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Preface 


This book is a collection of survey articles on some of the exciting recent devel- 
opments in algebraic combinatorics. It also contains a tutorial on Schubert calculus, 
as we felt that such an article would constitute an important addition to the 
literature. 

Algebraic combinatorics is a vast area of research, and attempting to be 
exhaustive in our coverage would have been a mistake. Rather (after reaching out to 
many experts in the field for advice), we chose a small number of topics that are 
currently enjoying broad interest and rapid growth. And after many thoughtful 
discussions, we sought out the authors of this volume—several writers known for 
their expository skills as well as a few junior researchers with active research 
programs who were willing to share their knowledge. Writing a good survey is a 
deceptively tricky endeavor, as decisions on what to include (or not) and how to 
best present it lurk at the corner of every sentence! Nonetheless, our contributors 
did a remarkable job of introducing exciting directions of current research in 
algebraic combinatorics from its foundational questions to the boundaries of what is 
known today. 

More specifically, the book contains four surveys focusing on representation 
theory, symmetric functions, invariant theory, and the combinatorics of Young 
tableaux. The other five surveys address subjects at the intersection of algebra, 
combinatorics, and geometry: the study of polytopes, lattice points, hyperplane 
arrangements, crystal graphs, and Grassmannians. The surveys are written at an 
introductory level that emphasizes recent developments and open problems. The 
tutorial on Schubert calculus is written in an interactive way and is intended as a 
guide for combinatorialists wishing to understand and appreciate the geometric and 
topological aspects of Schubert calculus, as well as for geometric-minded 
researchers seeking to gain familiarity with the relevant combinatorial tools in 
this area. 

Each article in this volume was reviewed independently by two referees, and we 
are simply amazed by, and deeply grateful for, the generosity of those referees. 
Their care in reviewing the articles and their constructive and judicious suggestions 
have been invaluable. The authors of this volume wish to warmly thank the referees 
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for their contributions, and we three editors in turn offer our heartfelt thanks to both 
the authors and the referees. We are very proud to have edited such a fine book. 

We hope that you will enjoy reading these articles as much as we have, and (as 
one referee wrote) you will find that the stories flow well and are inspiring! 


Berkeley, USA Héléne Barcelo 
Claremont, USA Gizem Karaali 
Hanover, USA Rosa Orellana 
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Partition Algebras and the Invariant ®) 
Theory of the Symmetric Group sieck 


Georgia Benkart and Tom Halverson 


Abstract The symmetric group S,, and the partition algebra P;(m) centralize one 
another in their actions on the k-fold tensor power Me of the n-dimensional permu- 
tation module M, of S,,. The duality afforded by the commuting actions determines 
an algebra homomorphism ®,;,, : Px(n) — Ends, (M2) from the partition algebra 
to the centralizer algebra Ends, (M®*), which is a surjection for all k,n € Zs, and 
an isomorphism when n > 2k. We present results that can be derived from the dual- 
ity between S,, and P;(n), for example, (i) expressions for the multiplicities of the 
irreducible S,,-summands of Me , Gi) formulas for the dimensions of the irreducible 
modules for the centralizer algebra Ends, ( Me*), (iii) a bijection between vacillat- 
ing tableaux and set-partition tableaux, (iv) identities relating Stirling numbers of 
the second kind and the number of fixed points of permutations, and (v) character 
values for the partition algebra P,(m). When 2k > n, the map ®;,, has a nontrivial 
kernel which is generated as a two-sided ideal by a single idempotent. We describe 
the kernel and image of ®;,, in terms of the orbit basis of P;,() and explain how 
the surjection ®, ,, can also be used to obtain the fundamental theorems of invariant 
theory for the symmetric group. 


Keywords Symmetric group - Partition algebra - Schur—Wey] duality 
Invariant theory 
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1 Introduction 


Throughout we assume F is a field of characteristic zero. The symmetric group 
S, has a natural action on its n-dimensional permutation module M,, over F by 
permuting the basis elements. The focus of this article is on tensor powers M&* of 
M,,, which are S,,-modules under the diagonal action (see (1.2)). For n,k € Zs, 
the partition algebra P; (7) is an associative F-algebra with basis indexed by the set 
partitions of {1,2,..., 2k} and with multiplication given by concatenation of set- 
partition diagrams (described in Sect. 4). In [22], V. Jones constructed a surjective 
algebra homomorphism 


Pky: Py(n) > Ends, (M®*) (1.1) 


from the partition algebra onto the centralizer algebra Ends, (M®*) of transformations 
that commute with the action of S, on Me. When n > 2k, this surjection is an 
isomorphism. 

Schur—Wey] duality relates the representation theory of a group and its central- 
izer algebra acting simultaneously on a tensor power representation (see, e.g., [16, 
Chap.9] or [19, Sects.3 and 5]). The classical case considers the general linear 
group GL, and its fundamental module M,, = F”. The centralizer of GL, on the ten- 
sor power M®* is provided by the surjection FS; > Endg., (M®*), where S; acts by 
permuting the tensor factors. This map is an algebra isomorphism whenn > k. Idem- 
potents in the group algebra FS; determine the projection maps onto the irreducible 
GL,-summands of M®*, and this enabled Schur to construct all the irreducible poly- 
nomial representations of GL, from the tensor powers M®*. A similar surjective 
algebra homomorphism, B;(”) — Endo, (M2) from the Brauer algebra B,(n) to 
the algebra Endo, (M®*) of transformations of M®* commuting with the action of 
the orthogonal group O,,, is an isomorphism when n > k and has been used to study 
the action of O, on M®*. When restricted to the subgroup of permutation matrices 
in O,, the module M, becomes the permutation module of S,,, and the inclusions 
S; C O, C GL, imply reverse inclusions of their centralizer algebras. In particular, 
when n > 2k, the centralizer chain is P,(n) D By(n) D FS,. In this duality picture, 
there are two different symmetric groups acting on M®*, namely the group S,, which 
acts diagonally and the group S; which acts as tensor place permutations. 

The Schur—Weyl] duality afforded by the homomorphism @,.,,, in (1.1) between 
the partition algebra P;,() and the symmetric group S,, in their commuting actions 
on Me enables information to flow back and forth between S,, and P;(n). Indeed, 
the symmetric group S, has been used to 


e develop the combinatorial representation theory of the partition algebras P; (7) as 
k varies [3, 4, 13, 14, 17-19, 28, 29] and 
e study the Potts lattice model of interacting spins in statistical mechanics [26-28]; 


while the partition algebra P;(m) has been used to 


e study eigenvalues of random permutation matrices [14], 
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e prove results about Kronecker coefficients for S,-modules [6-8], 

e investigate the centralizer algebras of the binary tetrahedral, octahedral, and icosa- 
hedral subgroups of the special unitary group SU, acting on tensor powers of 
V = C? via the McKay correspondence [1], and 

e construct a nonhomogeneous basis for the ring of symmetric functions and show 
that the irreducible characters of S, can be obtained by evaluating this basis on 
the eigenvalues of the permutation matrices, which has led to results on reduced 
Kronecker coefficients [31]. 


The irreducible modules S* for the symmetric group S,, are indexed by partitions 
A of n, indicated here by 4 n. The n-dimensional permutation module M,, for 
Sn has basis elements v,, v2, ...,V,, Which are permuted by the elements o of S,,, 
O.Vi = Voi). The module M,, has adecomposition into irreducible S,,-modules, M, = 
Si] @ Slr. where the vector v; + v2 +--+-+V, spans a copy of the trivial S,,- 
module sin], and the vectors v; — vj41 for 1 <i <n-—=1 form a basis for a copy 
of the (n — 1)-dimensional module S!"~':!, The module S!"~':!! is often referred 
to as the “reflection representation,” since the transposition switching i and j is a 
reflection about the hyperplane orthogonal to v; — v;. The S,-action on M,, extends 
to give the tensor power Me the structure of an S,,-module: 


oO. (vi, ® Vin ®---® Viz) = ON, ® ONiy ®::-@ ONi, = Vo(i,) ® Vo (i2) ®---@ Vo (ix): 
(1.2) 


In Sects.2 and 3, we describe two different ways to determine the multiplicity 
m{., of S/; in the decomposition 


@k _ ay vy 
Mi _ QB Men Si, 


Abn 


of Me into irreducible S,,-summands. The first approach, using restriction and induc- 
tion on the pair (S,,, S,,_1), leads naturally to a bijection between the set of vacillating 
k-tableaux of shape A and the set of paths in the Bratteli diagram ‘B(S,,, S,—1) from 
the partition [”] at the top of B(S,,, S,_1) to A at level k. Both sets have cardinality 
ue The second way, adopted from [4], uses permutation modules for S,, and results 
in an exact expression for the multiplicity of an irreducible S,,-summand of Me .In 
particular, we describe how [4, Thm. 5.5] implies that 


n 
k 
x A/[a- 
m= > ify Mn} 


t=|a*| 


where the Stirling number of the second kind ‘ea! counts the number of ways to 


partition a set of k objects into t nonempty disjoint blocks (subsets), f*/!"~" is the 
number of standard tableaux of skew shape A/[n — t], and 4° is the partition obtained 
from A by removing its first (largest) part. Therefore, 
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a Ge 3) m is a set partition of {1,2,...,k} into t parts, where |A*| <t<n 
Kn , S is a standard tableau of skew shape 4/[n — f] , 


In Sect. 3.2, we consider set-partition tableaux (tableaux whose boxes are filled 
with sets of numbers from {0, 1,...,k}) and demonstrate a bijection between set- 
partition tableaux and vacillating tableaux. Different bijections between these sets 
are given in [10, 18]. However, the bijections in those papers apply only when 
n > 2k. The bijection here has the advantage of working for allk, n € Z 1. Such set- 
partition tableaux also appear in the recent investigations of Orellana and Zabrocki 
[32, Example 2]. 

At the core of these results is the duality between the representation theories of 
the symmetric group S, and the centralizer algebra, 


Zin = Ends, (MP*) = {y € End(M?"*) | go(x) = og(x), o € Sp, x € MP}, 
(1.3) 
of its action on M®*. Schur—Wey] duality tells us that the irreducible modules Zin for 
the semisimple associative algebra Z;_, are indexed by the subset A;.5, of partitions 
i of n such that S* occurs with multiplicity at least one in the decomposition of Me 
into irreducible S,,-summands. Moreover, 


cM = (ms, = 7a; (1.4) 
LEAK Sp AEALSp 
ee - 
as an S, -module as aZ,_,-module 
e dim(Zi,) =m, (the multiplicity of S* in M®*); (1.5) 
° mult(Z¢.,) = dim(S?) =f + (the number of standard tableaux of shape A); 
(1.6) 
« dinZ)=— > (mi, =my), =dim(Zy,). (1.7) 
AC AL Sy 


The first equality in (1.7) is a consequence of (1.5) and Artin—Wedderburn the- 
ory, since dim(Z;,,) is the sum of the squares of the dimensions of its irre- 
ducible modules Zi The second equality in (1.7) comes from the isomorphism 
Zin = Ends, (M®) = (M2*)S, since the S,-invariants in M®" correspond to 
copies of the trivial one-dimensional module $!"! indexed by the one-part partition 
[n], and my), = dim (Z4},,) is the number of trivial summands of M2”. 

Three additional Schur—Wey] duality results can be found in [5, Cor. 2.5]. From 
those results, we know that for any finite group G and any finite-dimensional G- 
module V, the dimension of the space (ve")S of G-invariants is the average of 
the character values xyex(g) = xy (g)* as g ranges over the elements of G (com- 
pare also [15, Sect. 2.2]). When V is isomorphic to its dual G-module, the dimen- 
sion of the centralizer algebra Endg(V®) is the average of the character values 
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xvex(g) = x, (g)**. Specializing those results and part (i) of [5, Cor. 2.5] to the case 
G = S, and V = M, gives the following: 


e =6 dim ((me*)” >> xu, (o)* = > F(a); (1.8) 
. ae : Paes. 
1 
e == dim(Z,,,) = dim(Z4_) = — > F(a)"; (1.9) 
, nN: 
oS, 


1 1 
© dim(Zé,) = | D) xm, (0)' (o7') = | DF) xx(0), 1-10) 


oS, * o€S, 


where xm, (o) = F(o) (the number of fixed points of 0) and x, is the character of 
the irreducible S,,-module S$‘. In (1.10), we have used the fact that o and o~! have 
the same cycle structures, hence the same character values. 

In [14], Farina and Halverson develop the character theory of partition algebras 
and use partition algebra characters to prove (1.9). The arguments in [14], which are 
based on results from [17], require the assumption n > 2k so that Z; , = P;,(m). The 
results in (1.8)—(1.10) are true for all k,n > 1. 

When V is a finite-dimensional module for a finite group G, the tensor power V* 
has a multiplicity-free decomposition 


v% = GD (G" @ Z;.) 


v 


into irreducible summands as a bimodule for G x Z;.¢, where Z;.¢ = Endc(V™). 
Consequently, the characters of G, Z;,g andG x Z,;.¢ are intertwined by the following 
equation, 


Wver(g x 2) = D> xv(g) i). 


The irreducible characters of G are orthonormal with respect to the well-known 
inner product on class functions of G defined by (a, 8) = |G|~! Donec a(g)B(g7!) 
(see, e.g., [15, Thm. 2.12] or [35, Thm. 1.93]). Therefore, since wye(- x z): G> F 
is a class function on G for each z € Z;.¢, we have 


(Wrvor(- X 2), Xv) =) wae x z)xXv(g7") = &y(2). 


geG 


~ 1G] 


Thus, the commuting actions of G and Z;.¢ on V® lead to one further Schur—Wey] 
duality result, namely an expression for the irreducible characters &, of Z; ¢: 


© &()= qe Wale x 2xo(e (1.11) 


geG 


www.TechnicalBooksPDF.com 


6 G. Benkart and T. Halverson 


In Sect.4.5, we explain how these ideas, when combined with results from [17], 
provide the formula in Theorem 4.17 for the characters of the partition algebra P; (7). 

The surjective algebra homomorphism ®,, : Px(n) > Zen = Ends, (M®*) 
enables us to study the S,,-module M®* using the partition algebra. Since the set 
partitions of the set [1, 2k] := {1,2,..., 2k} index an F-basis for P;, (1), the image 
of these basis elements linearly spans the S,-module endomorphisms Ends, (M®*), 
and the generators of P;(m) generate Ends, ( Me) as an associative algebra. 

We let IT; be the set of set partitions of [1, 2k]; for example, 1 7,8, 10/|2,5| 
4,9,11| 3,6, 12, 14| 13} is a set partition in [7,4 with 5 blocks (subsets). The 
number of set partitions in [7>, with t blocks is given by the Stirling number of the 
second kind ey }, and thus P;(m) has dimension equal to the Bell number B(2k) = 
Da) 

The algebra P, () has two distinguished bases—the diagram basis {d, | w € Ix} 
and the orbit basis {x, | 2 € II2,}. We describe the change of basis matrix between 
the diagram basis and the orbit basis in terms of the Mobius function of the set- 
partition lattice (see Sect. 4.3). Section 4.4 is devoted to a description of multiplication 
in the orbit basis of P;(”). The actions of the diagram basis element d, and the orbit 
basis element x, on Me afforded by the representation ®,, are given explicitly 
in Sect.5.2 for any 2 € IIn,. The orbit basis is key to understanding the image and 
the kernel of ®;.,, as we explain in Sect.5.3. An analog of the orbit basis exists in 
a broader context, and in Sect.4.2 we explain how to construct an orbit basis for 
the centralizer algebra of a tensor power of any permutation module for an arbitrary 
finite group G. 

The centralizer algebra Z;.,, = Ends, (M®*) has a basis consisting of the images 
P, , (Xz), where z ranges over the set partitions in [7,; with no more than n blocks. 
As a consequence of that result and (1.9), we have 


"(2k 1 
dim(Zy n) = >| ; =F > F(a)**. (1.12) 


t=1 oéS, 


We prove additional relations between Stirling numbers of the second kind and fixed 
points of permutations in Sect.3.1 (more specifically, see Theorem3.7, Proposi- 
tion 3.9, and Corollary 3.11). 

The diagram basis elements d,, in Px, ,1(”) corresponding to set partitions z having 
k + 1 and 2(k + 1) in the same block form a subalgebra P,., 1 (n) of Py) (7) under 
diagram multiplication. Identifying P;,() with the span of the diagrams in P,., 1(n) 
that have a block consisting solely of the two elements k + 1, 2(k + 1) gives a tower 
of algebras, 


F = Po(n) = Pi(n) CPi(n) C-++ C Pe() C Pyyi(n) C Peyi(m) Cee: . 
(1.13) 
If we regard M,, as a module for the symmetric group S,_; by restriction, where 
elements of S,,_; fix the last basis vector v,,, then there is a surjective algebra homo- 
morphism 
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: @k 
Py. 1, 2 Pea i(n) > Ends, (M2), (1.14) 


which is an isomorphism when n > 2k + 1. The intermediate algebras P,, 1 (7) have 
proven useful in developing the structure and representation theory of partition alge- 
bras (see, e.g., [19, 29]), and they come into play here in the construction of vacil- 
lating k-tableaux. The centralizer algebras Ends,_, (M®*) are also closely tied to the 
restriction and induction functors that produce the Bratteli diagram B(S,,, S,—1) in 
Sect. 2.3. 

For a group G and a finite-dimensional G-module V, the tensor product V ® V* 
of V with its dual module V* is isomorphic as a G-module to End(V) via the 
mapping v@@t> Avg, where Ay. g(u) = d(u)v. Since g.(v@o) = g.v®g.d, 
where (g.¢)(u) = (g~!.u), and g.A = gAg™! as transformations on V for all 
A € End(V), we have 


g(V@d)=VvOd => gdavgg | = Avo gAv.o = Av.68 Ay, € Endg(V). 


Thus, the space of G-invariants (V @ V*)° = {v@@|g.(v@@¢) =v ® 9} is iso- 
morphic to the centralizer algebra Endg(V). 

In the particular case of the symmetric group S,, and its permutation module M,, 
we can identify the centralizer algebra Ends, (M®*) with the space 


(M24) = (Me* @ (Me%*)> 


of S,,-invariants, as M,, is isomorphic to its dual S,,-module M* (this was used in 
(1.7) and (1.9)). The fact that ®,, : Py(m) > Ends, (M2) is a surjection tells us 
that P;,(7) generates all of the S,,-tensor invariants ( M22k)Sn and provides the First 
Fundamental Theorem of Invariant Theory for S,, (see Theorem 6.1 originally proven 
by Jones [22]). When 2k > n, the surjection ®,, has a nontrivial kernel. As 2k 
increases in relation to n, the kernel becomes quite significant, and Ends, (M2) is 
only ashadow of the full partition algebra (This is illustrated in the table of dimensions 
in Fig. 5.). We have shown in [3] that when 2k > n, the kernel of ®;,, is generated 
as a two-sided ideal by a single essential idempotent e; , (see (5.20)). Identifying 
Ends, (M®*) = (m22e)> with P,(n)/ker ®y.n (= Px(n)/(ex.n) when 2k > n), we 
have the following. 


Theorem 1.15 [3, Thm. 5.19] (Second Fundamental Theorem of Invariant Theory 
for S,) For all k,n € Zs\, im ®y, = Ends, (M2*) is generated by the partition 
algebra generators and relations in Theorem 6.5(a)-(c) together with the one addi- 
tional relation ex, = 0 in the case that 2k > n. When k => n, the relation ex, = 0 
can be replaced with e,,, = 0. 


Classical Schur—Weyl duality provides an analogous result for the general linear 
group GL,, and its action by matrix multiplication on the space V = M, = F” ofn x 1 
column vectors (more details can be found in [16, Sect. 9.1]). The surjection FS; > 
Endc., (V®*), given by the place permutation action of S; on the tensor factors of V, 
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is an isomorphism if n > k, and in that case, FS; = (ver @ (Vk )*) Se When k > 
n + 1, the kernel is generated by a single essential idempotent )°, se a3 (-1L)2"© a 
in the group algebra FS;. Thus, the second fundamental theorem comes from the 
standard presentation by generators and relations for the symmetric group S; by 
imposing the additional relation )>,25,,, (— "oo = 0 whenk >n +1. 

In [9], Brauer introduced algebras B;(), now known as Brauer algebras, that 
centralize the action of the orthogonal group O, and symplectic group Sp,,, on 
tensor powers of their defining modules. The surjective algebra homomorphisms 
By(n) > Endo, (V®) and By(—22) > Endsp, (V®") (where V = M,, for O, and 
V = Mz», for Sp,,,) defined in [9] provide the First Fundamental Theorem of Invariant 
Theory for these groups (An exposition of these results appears in [16, Sect. 4.3.2].). 
Generators for the kernels of these surjections give the Second Fundamental Theorem 
of Invariant Theory for the orthogonal and symplectic groups. In [20], Hu and Xiao 
showed that the kernel of the surjection in the symplectic case is principally generated 
when k > n+ 1. Lehrer and Zhang [23, 24] identified a generator for the kernel 
in both the symplectic and orthogonal cases. The Brauer diagram category is an 
essential ingredient in these papers (and also in the symplectic group investigations 
of Rubey and Westbury [33, 34]) for proving that the kernel of the surjection is 
principally generated and in establishing analogous results for the quantum analog of 
the Brauer algebra, the Birman—Murakami—Wenzl algebra. As shown in [24, Lemma 
5.2] (compare [33, Sects. 7.4 and 7.6] and [34] which establish connections with 
cyclic sieving), the element FE = au >>, 4, obtained by summing all the Brauer 
diagrams d with 2n + 2 vertices, is acentral idempotent in B,,,; (—2n) corresponding 
to the one-dimensional trivial B,,+;(—2n)-module, and it generates the kernel of the 
surjection By(—2n) — Endsp,, (V®) for allk > n+ 1. As aresult, the fundamental 
theorems of invariant theory for the symplectic and orthogonal groups can be obtained 
from Brauer diagram considerations. 

Bowman, Enyang, and Goodman [8] adopt a cellular basis approach to describing 
the kernels in the orthogonal and symplectic cases, as well as in the case of the 
general linear group GL, acting on mixed tensor powers V®* @ (V*)®* of its natural 
n-dimensional module V and its dual V*. A surjection of the walled Brauer algebra 
Bz.e(n) > Endg, (V@ @ (V*)®*) is used for this purpose (The algebra Bx,¢(n) and 
some of its representation-theoretic properties including its action on V@* @ (V*)®¢ 
can be found, e.g., in [2].). 

Although this article is largely a survey discussing recent work on partition alge- 
bras and the fundamental theorems of invariant theory for symmetric groups, it 
features new results. Among them are a new bijection between vacillating tableaux 
and set-partition tableaux, a new expression for the characters of partition algebras, 
and new identities relating Stirling numbers of the second kind and fixed points of 
permutations. 

Our main point of emphasis is that the Schur—Wey] duality afforded by the surjec- 
tion Dy, : Pe(n) > Ends, (M2) furnishes an effective framework for studying the 
symmetric group S, and its invariant theory. Here n is fixed, and the tensor power 
k is allowed to grow arbitrarily large. When 2k > n, the kernel of ®; ,, can best be 
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described using the orbit basis of the partition algebra and can be used to give the 
Second Fundamental Theorem of Invariant Theory for S,,. 


2 Restriction—Induction Bratteli Diagrams and Vacillating 
Tableaux 


In this section, we discuss the restriction and induction functors for a group—subgroup 
pair (G, H) and the Bratteli diagram that comes from applying them and then spe- 
cialize to the case of the pair (S,,, S,_1). Further details can be found, for example, 
in [4, Sect. 4]. 


2.1 Generalities on Restriction and Induction 


Let (G, H) be a pair consisting of a finite group G and a subgroup H of G. Let U® be 

1 
the trivial one-dimensional G-module. For k € Zs, construct the H-module Uk+3 
by restricting U* to H and then construct the G-module U‘*! by inducing Uk+? to 


G. Thus, 
Uk? = ResG(U‘) and US! = IndS(U‘+7) = FG @py U2. (2.1) 
In this way, U* is defined inductively for all £ € 5Zz0, and 
Uk = (Ind§Res$)“(U°) for allk € Zzo. (2.2) 
In particular, the module M := Indg (Resg(U°)) = U! is isomorphic to G/H as a 
G-module, where G acts on the left cosets of G/H by multiplication. 
For a G-module X and an H-module Y, the “tensor identity” says that 
Ind (Res§(X) @ Y) = X @ Ind§(Y). (2.3) 
The mapping (g @FH x) © y +> gx @ (g @pu y) can be used to establish this iso- 
morphism (see, e.g., [19, (3.18)]). Hence, when X = U‘ and Y = Res¢(U®), (2.3) 
implies that 
Indg (ResG (U*)) = Ind& (Res (U*) @ Resg(U°)) = UK @ Ind& (Res§(U°)) = UK @ M. 


Therefore, by induction, we have the following isomorphisms for all k € Zo: 


Me = Uk (asG-modules) and  Resg(M®*) = Uk+2 (as H-modules). 
(2.4) 
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The centralizer algebra of the G -action on M®* is defined as 
Zx.¢ = Endg(M®*) = {y € End(M®*) | g(g.x) = g.g(x) for all x € M®*, g € G}, 


and the centralizer algebra for the restriction to H is denoted by Z,, y= 


Endy (Resg(M®*)). As a consequence of (2.4), we have algebra isomorphisms 


Zig = Endg(M®) & Endg(U*) and 


(2.5) 
Zi41.4 = Endy(Res§(M@*)) & Endy(U'*?). 

For k € Zs we adopt the following notation to index the irreducible summands in 
these modules. 


e Axg © Ag indexes the irreducible G-modules, and hence also the irreducible 
Zx,G-modules, occurring in Uk = Me; 
@ Ay Ly C Ap indexes the irreducible H-modules, and hence also the irreducible 


ee dts 
Z;.4.1,4-modules, occurring in Uk+2 & ResG(M®*), 


2.2. The Restriction-Induction Bratteli Diagram 


The restriction—induction Bratteli diagram for the pair (G, H) is an infinite, rooted 
tree B(G, H) whose vertices are organized into rows labeled by half integers @ in 
5Zz0. For € =k € Zso, the vertices on row k are the elements of A; ¢, and the 
vertices on row £ =k + 5 are the elements of A,,1 4. The vertex on row 0 is the 


root, the label of the trivial G-module, and the vertex on row 5 is the label of the 
trivial H-module. 

The edges of B(G, H) are constructed from the restriction and induction rules for 
H C Gas follows. Let {G*}, ca, and {H}ye4, be the sets of irreducible modules 
for these groups over F. By Frobenius reciprocity, the multiplicity c* of H® in the 
restricted module Res¢ (G*) equals the multiplicity of G* in the induced module 
Indg(H®), and thus 


Resi(G*) = Gch H® and —Indg(H*) = G) ch G’. (2.6) 


aeAy AE AG 


In B(G, H), we draw c* edges from A € Ag.g toa € Agyt, y and from @ € Ayn 
tor € Az+i,¢. Hence, the Bratteli diagram is constructed in such a way that 


e The number of paths from the root at level 0 to A € Axz,g equals the multiplicity of 
G* in UK = M® and thus also equals the dimension of the irreducible Z;,¢-module 
Le by (1.5) (these numbers are computed in Pascal-triangle-like fashion and are 
placed beneath each vertex); 
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e The number of paths from the root at level 0 to@ € A;, Ly equals the multiplicity 


of H® in U*+? and thus also equals the dimension of the Z,, 1 4-module Lie ng 
(and is indicated below each vertex); : 
e The sum of the squares of the labels on row k (resp. row k + 5) equals dim(Z,.¢) 


(resp. dim(Z,, 1 44). 


The restriction—induction Bratteli diagram for the pair (Ss, S4) is displayed in Fig. 1. 


2.3 Restriction and Induction for the Symmetric Group Pair 


(Sn ) Sn-1) 


Assume A = [Aj,A2,..., An] 18 a partition of n with parts A; > Ag >... = Ay = O* 
and identify 2 with its Young diagram. For example, we identify the partition A = 
[6, 5,3, 3] F 17 with its Young diagram as follows, 


A = [6, 5, 3,3] = 


The hook length h(b) of a box b in the diagram is | plus the number of boxes to the 
right of b in the same row, plus the number of boxes below b in the same column. 
Thus, )(b) = 1+ 3+ 2 = 6 for the shaded box in the example above. The dimension 
f* of the irreducible S,,-module S* can be computed by the hook-length formula, 


n! 


2: | 
PT FO’ a 
where the denominator is the product of the hook lengths as b ranges over the boxes 
in the Young diagram of A. This is equal to the number of standard Young tableaux 
of shape A, where a standard Young tableau T is a filling of the boxes in the Young 
diagram of 4 with the numbers {1, 2,...,} such that the entries increase in every 
row from left to right and in every column from top to bottom. 

The restriction and induction rules for irreducible symmetric group modules S* 


are well known (and can be found, e.g., in [21, Thm. 2.43]): 


Rese” (S}) = QB 57-4 Inder! (S*) = QB eee (2.8) 


p=)r— K=A+ 


where the first sum is over all partitions 44 of n — 1 obtained from A by removing 
a box from the end of a row of the diagram of A, and the second sum is over all 
partitions « of n + | obtained by adding a box to the end of a row of i. 
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Applying these rules to the trivial one-dimensional S,,-module S!”!, we see that 
Sn Sn Sn -1 -ll]~ 
Indg"_,(Res=*_,(S4"")) = Inde" , (Si) = Si"! @ Sth" = M,,. 


Thus, in the notation of Sect.2.1, U! is the permutation module M,,, and by (2.4), 


Me = U* (asS,-modules) and Res2” _ (MPS) = Uk+3 (as S,,_1-modules). 


(2.9) 

Then (2.5) implies that the centralizer algebras are given by 
Zin c= Ends, (M®) = Ends, (U*), (2.10) 
Zy+19 += Ends,_,(M®*) = Ends, _,(U*), (2.11) 


where we are writing Z;,, rather than Z;5, and Z,;, 1 thn instead of Z, +15, to simplify 
the notation. (The use of n in place of the more natural choice of n — 1 for the second 
one should be especially noted.) 
From our discussions in Sects. 2.2 and 2.3, we know that the following holds. 
Ifk,n € Zso andn > 1, then for alld € Ax s,, 


dim(Z{.,,) = mz, = |{ paths in B(S,,, S,1) from [n] at level 0 tod at level k}]. 
(2.12) 

In Fig. 1, we display the first several rows of the restriction—induction Bratteli 
diagram B(Ss, S4). Below each partition, we record the number of paths from the 
top of the diagram to that partition. For Me p SE values of k, the number beneath 
the partition A + n represents the multiplicity m% Xn Of the eeu. Sn-module S* 
in M®* (with n = 5 in this particular example). For values k +5 5, it indicates the 
multiplicity of S“_,, «4 + n — 1, in the restriction of M&* to S,-1. 1. The number on 
the right of each line is dim(Z;,,) (or dim(Z,, 1 nd)» Which is the sum of the squares 
of the subscripts on the line. 

The module M,, is faithful, so by Burnside’s theorem every irreducible S,,-module 
appears in M®* for some k > 0. In the example of Fig. 1, all of the irreducible Ss- 
modules appear in me (i.e., all of the partitions of 5 appear in B(Ss5, S4) at level 
£ = 4). More generally, we can use the Bratteli diagram to establish the following 
result. 


Proposition 2.13 (i) The irreducible module S*, n, appears as a summand of 
Me if and only if k = \d*| (where X* is the partition obtained by removing the 
first part of 2). 

(ii) All irreducible S, modules appear in M®* if and only if k > n — 1. Thus, 


Ags, = As, ={A|AE n} ifandonlyif k>n—1. (2.14) 


Proof (i) Let A be a partition of n. Successively remove the last box from the bottom 
row and place it at the end of the first row until the partition has only one row. 
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a4 | 

in : 
ewan : 
, i : 
a Oe ae ae . 
IN AYSKEXIN ; 
ALON XE 
IN ASXINNY | 
AL NLR XI 
- le ae Gs die _ 


52 150 145 156 105 


Fig. 1 Levels 0, 5s Ds assay 35, 4 of the Bratteli diagram for the pair (S5, S4). All of the partitions 
of 5 appear on level ¢ = 4, and the structure of the diagram stabilizes for ¢ > 4 


Counting the removal of a box as one step and the adjoining of the box to the first 
row as another, this sequence of partitions, when read in reverse order, determines a 
path in B(S,, S,—1) from [n] at level 0 to A at level |A*|. Thus, S* has multiplicity at 
least one in mel, For example, the path to 7 = [3, 2, 1] at level |A*| = |[2, 1]| =3 
in B(S6, Ss) given by this construction is 
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It is impossible for 2 to appear before level |A*|, since any path from [n] to A 
requires removing |A*| boxes from the first row of the partition and placing them in 
lower rows. Finally, if A € Ax,s, for some integer k > 0, then A € Ax+1,5,, because 
we can always remove any removable box in A and then place it back in its same 
position. Therefore, S* occurs in M® if and only if k > |A*]. 

(ii) As k increases, the last partition of n to appear in Me is A = [1”], which by 
part (i) first occurs when k = |[1”"]*| =n — 1. Thus, k > n — 1 is a necessary and 
sufficient condition for As, to equal As, . 


Paths in the Bratteli diagram B(S,, S,—1) give the dimension Mn of the irre- 
ducible module 7, of the centralizer algebra Z;,, in the same way that paths in 
Young’s lattice [35, Sect. 5.1] give the dimension of the symmetric group module 
se For this reason, we make the following definition (see also [4, 10, 18], where 
this definition is used). 


Definition 2.1 Letk € Zo and letA € A,.s,. Each path of length k from [7] at level 
0 to 4 at level & in the Bratteli diagram B(S,,, S,_1) determines a vacillating tableau 
v of shape X and length k, which is an alternating sequence 


v= (29 =f), 2® = fe— 11, 0M, 007, ...,a0-D® a) 


of partitions starting at A© = [n] and terminating at the partition A“ = A such that 
“ ey l . . 
WM € Ajs,, AD Aj1\5,., for each integer 0 <i < k, and 


> 
f 


(a) GD) = 29 —O, 
(b) AGH) = ala) 


In Sect. 3.3, we describe a bijection between vacillating tableaux of shape A and 
length k and set-partition tableaux of shape 4 whose nonzero entries are 1,2, ..., k. 
This bijection is analogous to the well-known bijection between paths to A of length 
k in Young’s lattice and standard tableaux of shape 4 | k (see [35, Sect. 5.1]). Since 
paths in the Bratteli diagram correspond to vacillating tableaux, (2.12) tells us the 
following: 

If k,n € Zso and n > 1, then forall r € As, 


in? 


dim(Z;,,,) = Men I | { vacillating tableaux of shape 4 and length k} |. (2.15) 
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3 Set Partitions 


For k € Zs, let [1, 2k] = {1, 2,..., 2k}, as before. The set partitions in 


Io, = {set partitions of [1, 2k]}, 


TTog_, = {0 © Ty, | k and 2k are in the same block of zr} . GBA) 


index bases of the partition algebras P;,() and P,, 1 (n), respectively. We let |z'| 
equal the number of blocks of x. For example, if 


mx = {1,8,9, 10|2,3|4,7|5, 6, 11, 12, 14] 13} © Ma, 


po = 41,8,9, 10|2,3|4]5, 6, 11, 12|7, 13, 14¢ € M3 C Ma, A) 
then wz ¢ [73 and |z| = |p| =S. 
For £,n € Zs, define 
TT; , = {a € II, | a has at most n blocks}. (3.3) 


The cardinalities of these sets are Bell numbers: |/7¢| = B() and |J7;,,| = B(é, 7). 
We refer to B(¢, n) = )7"_{ y as an n-restricted Bell number. 


3.1 Multiplicities from a Permutation Module Perspective 


We begin by discussing a second approach to decomposing M®* using permutation 
modules for S,, and then describe connections with set-partition tableaux. 

Let {v1, v2, ..., Vn} be the standard permutation basis of M,, such that o0.v; = voi) 
foro € S,. Assume z isa set partition of {1, 2,..., k}intot blocks, where 1 <t <n. 
Then the vector space 


M(t) := spane fi @Vjy @ ++ OVix | ja = jb <> a,b are in the same block of x} 
(3.4) 
is an S,,-submodule of Me . To see this, recall that 0 € S,, acts diagonally on simple 
tensors, 0.(v;, @ Vj, @ +--+ @ Vj) = Vo(j,) ® Voip) ® +++ ® Vor j,), and so it preserves 
the condition in (3.4). As an example, ifm = 8 and k = 12, then 


V = V3 @ Vi @ V3 @ V3 @ V4 @ V5 B V4 @ Vi W Vi @ V3 @ V4 W V5 € Met (3.5) 


belongs to M(zr) for wm = {1, 3, 4, 10 | 2,8,9| 5,7, 11 | 6, 12}. 

A simple tensor v = vj, ® vj, ®--- ® vj, € M(zr) also determines an ordered set 
partition of the subscripts {1, 2, ..., 1} into blocks of size [n — t, 1] in which the 
t distinct subscripts of v are in blocks by themselves, in the order they appear in 
v, and the n — ¢ unused subscripts are in the first block. In the simple tensor v of 
(3.5), there are tf = 4 distinct subscripts (namely, 3, 1,4, and 5, in that order) and 
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n—t =8—4= 4 unused subscripts {2, 6, 7, 8}. Such ordered set partitions can be 
represented by the rows of a tabloid of shape [n — t, 1'] (see [35, Sect. 1.6, 2.1] for 
details on tabloids). In this particular example, the corresponding tabloid is 


(2,6,7,8|3|1|4|5}<> 2678. 
3 
a 
A 
2 


The diagonal action of S, on the simple tensors in M(zr) corresponds exactly to 
the permutation action of S,, on the ordered set partitions of {1,2,...,} of shape 
[n — t, 1'] or, equivalently, on the tabloids of shape [n — t, 1‘]. The span of these 
tabloids is the well-known permutation module M!"~“-") obtained by inducing the 
trivial module for the subgroup S,_; x S; x --- x S; (witht copies of S;) to S, (see, 
e.g., [35, Sect. 2.1]). Thus, 


Mor) = M'"-"!' when z has t blocks. 


Note that the sizes of the blocks of 2 are immaterial to this isomorphism, what 
matters is the number f of blocks. Also observe that since M,, = M!"—!:4), the use of 
the term “permutation module” for both is consistent. 

The number of set partitions z of the tensor positions {1,2,...,k}intol <t<n 
parts is the Stirling number i) of the second kind, so by partitioning the simple 
tensors this way, we obtain the following decomposition of M®* into permutation 
modules for S,: 


wD fk 
met = {Fale G.6) 


t=1 


Note that iy = 0 whenever ¢t > k. 

Young’s rule (see, e.g., [35, Thm. 2.11.2]) gives the decomposition of the permu- 
tation module MY for y = [1, y2,---, Yn] 4 n into irreducible S,,-modules se It 
states that the multiplicity of S* in M” equals the Kostka number K,,,. The Kostka 
number K,,,, counts the number of semistandard tableaux T of shape A and type y, 
where such a semistandard tableau 7 is a filling of the boxes of the Young diagram 
of 2 with the numbers {0”, 1”, ..., (2 — 1)”"}, and the entries of T weakly increase 
across the rows from left to right and strictly increase down the columns. It follows 
that 


me = QO | i Mine > QO | i bs Ks) = ‘= (>: | | Ke) si. 
t=1 t=1 


Aen AEn \t=1 
In this case, the Kostka number K,, ,—;,1:; counts the number of semistandard 


tableaux of shape A filled with the entries {0"~’, 1, 2,..., t}. A semistandard tableau 
of shape 4 whose entries are n — ¢ zeros and the numbers 1, 2, ..., ¢ must have the 
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n — t zeros in the first row and have a standard filling of the skew shape A/[n — tf] 
with the numbers 1, 2,..., ¢. In particular, if A = [7,5, 3] and t = 3 then one such 
semistandard tableau is 


010 /0/3|6/8I1 
eh 
219 10 


The number f*/!"~l of such fillings is given by the hook formula for skew shapes 
(see, e.g., [36, Cor. 7.16.3]). Furthermore, Ky fn—1,177 = 0 whenever A, < n — t. That 
is, Ka.[n—1,19) = 0 whenever t <n — A, = |A*|, where A* is the partition obtained by 
removing one copy of the largest part of 2. 

The next result combines [4, Thm. 5.5] with (1.8)—(1.10). In the statement, Me 
should be interpreted as being F = S!”! (as an S,,-module). The discussion above 
provides a proof of the final equality in part (a) of the following theorem. 


Theorem 3.7 Fork € Zso andn € Z 31, suppose Me = Driers, my aces Assume 
Zin is the irreducible module indexed by i for Zyn = Ends, (M®*), as in (1.4)-(1.6). 


(a) [fA € Axs,, then 


1 n k 
ni > F@)*.@) = din(Z, .) = mz, = > ("| print 


oeSy =|" | 


where F(c) is the number of fixed points of o, and f*/""—" is the number of 
standard tableaux of skew mre A/[n — t]. 


‘ =~ i 2k 
(b) dim(Z;,,) = dim (ZY) = a F(o)* = >| te B(2k, n). 
OGOn 


Furthermore, dim(Z, ,) = B(2k, n) = B(2k) ifn => 2k. 
(c) fue Artis,» then 


Y> (Ft) + 1)‘ xu (2). 


TESy-1 


k+1 n—1-t 1 
dim (Z;', 4 = De {' a fell 1 = i = 15! 


; a alg (hes ae ag | 
(d) dim(Z,,1,,) = dim (a) = | = B(2k+1,n). 
t=1 
Furthermore, dim(Z,., 1 ,,) = BQ2k+ 1,n) = Bk +1) ifn > 2k +1. 


Remark 3.8 Whenn > k, the top limit in the summation in part (a) can be taken to 
be k, as the Stirling numbers {t} are O for t > k. When n < k, the term [n — f¢, 1'] 
for t = n is assumed to be the partition [1”]. In that particular case, Ky pin) = f the 
number of standard tableaux of shape A, as each entry in the tableau appears exactly 
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once. When t = n — 1, the Kostka number is the same, Ky, p17] = f*. The only time 
that the term ¢ = 0 contributes is when k = 0. In that case, ey = |, and the Kostka 
number K, ;,; = Oif A A [n] and Kjyy fn; = 1. Thus, dim Zi (n) = 5) nj, as expected, 
since M2? = Sl"! 


In items (c) and (d) of Theorem 3.7, we have used the fact that as an S,,_;-module, 
M, = M,-1 @ Fv,, so the character value xy, (t) of t € S,-; equals 1 plus the 
number of fixed points of t viewed as a permutation of {1,..., — 1}. Then since 
Las iy = Ends, , (M2) and M, is self-dual as an S,,_;-module, the first line of part 
(d) holds. The expressions in (a) and (c) can be related by the next result. 


Proposition 3.9 For all k € Zso, 


1 
k+1 _ I [n] _ 4 [n] 
al Foy! = mpi, = dim (Z7),,) 
“oS, 


: n—- n— 1 k 
= dim Ce) = mr = aD! > (F(t) +1), 
TES] 


(3.10) 


[n—1] 
k+3,n-1 


[n] 


where m;.,,, is the multiplicity of SY) in M°“*), and m is the multiplicity 


of os in Me, viewed as an S,_,-module. 


Proof The equalities in the first line of (3.10) are a consequence of taking 1 = [n] 
in part (a) of Theorem 3.7, and the equalities in the second line come from setting 
ju equal to [n — 1] in part (c). Now observe that the only way S!”! can be obtained 
by inducing the S,,_ ;-module Rese” (Me) to S,, is from a summand Sue so the 
multiplicity of S! in M@4+) = Inde" (Res2" (M&*)), which equals dim (Zia) : 
is the same as the multiplicity of $7" in Rese” (M&*), which equals dim (Zoos ae 
This can also be seen in the Bratteli diagram B(S,,, S,_1), as there is a unique path 
between [n — 1] at level k + 5 and [n] at level k + 1 (compare Fig. 1). 


We know from Theorem 3.7(b) that the next result holds for even values of £. 
Proposition 3.9 combined with part (d) of the theorem allows us to conclude that it 
holds for odd values of £ as well. Consequently, we have the following: 


Corollary 3.11 For all € € Zso andn € Zs}, 


aie (Fs 1 
B(é,n) = ~~ : oo ‘a F(a)’. (3.12) 


t=0 “oS, 


Remark 3.13 The identity in (3.12) has connections with moments of random per- 
mutations. If the random variable X denotes the number of fixed points of a uniformly 
distributed random permutation of a set of size £ > | into no more than n parts, then 
the £th moment of X is 
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n 


e 1 
E(x) = ~ | = B(l,n) = 7 > F(a)’. 


t=1 “eS, 


This formula can be derived by applying Polya’s Theorem [25, I, 2, Ex. 9] or by 
using partition algebra characters as is done in [14, (18)] for n > 2k. 


Remark 3.14 The results of this section relate Stirling numbers of the second kind 
to the number of fixed points of permutations. In the case of parts (a) and (c) of The- 
orem 3.7, the expressions involve Stirling numbers and also the number of standard 
tableaux of certain skew shapes. It would be interesting to have combinatorial bijec- 
tions that demonstrate the identities. A determinantal formula for the number f*/” 
of standard tableaux of skew shape 4/v was given by Aitken (see [36, Cor. 7.16.3]). 
Hook-length formulas for f*/” are studied in [30] and the references therein. 


3.2 Set-Partition Tableaux 


Part (a) of the Theorem 3.7 has inspired the following definition: 


Definition 3.1 For 4 = [A;, A2,..., A,] a partition of n, assume A* = [Az,..., An] 
and t € Zis such that |A*| < t < n. A set-partition tableau T of shape and content 
{0"~', 1,..., k} is a filling of the boxes of A so that the following requirements are 
met: 


(i) the first n — t boxes of the first row of i are filled with 0; 

(ii) the boxes of the skew shape A/[n — 1] are filled with the numbers in [1, k] = 
{1,2,...,k} such that the entries in each box of A/[n — t] form a block of a set 
partition 2(T) of [1, k] having t blocks; 

(iii) the boxes of T in the skew shape A/[n — ft] strictly increase across the rows and 
down the columns of A/[n — t], where if b; and b> are two boxes of 4/[n — f], 
then b; < bz holds if the maximum entries in these boxes satisfy max(b,) < 
max(b). 


Example 3.15 Below is a set-partition tableau T of shape A = [5, 4, 2, 1] F 12 and 
content {0*, 1, 2,..., 20} with corresponding set partition 7(T) = {1, 6 | 4,7, 9, 10 | 
2, 11,12] 8, 14| 15, 16| 5, 13, 18 | 3,17, 19 | 20} € Tyo «consisting of t=8 
blocks. The blocks of (T) are listed in increasing order according to their largest 
elements, which are the underlined numbers. 


0 0 0 Wa 3,17, 19 
1,6 |4,7,9,10/5, 13,18| 20 

T= \o 91,10) 8.14 
15, 16 
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Remark 3.16 If) is a partition of n with |A*| = k and T is a set-partition tableau of 
shape A and content {0"~‘, 1, ... , k}, then the k boxes of T corresponding to A* form 
a set partition of [1, k], so they must be a standard tableau of shape 4. The first row of 
T must then consist of a single row of zeros of length of t = 4; =n — |A*| =n —k. 
For example, if n = 9 and k = 5, then one possible set-partition tableau of shape A 
with A* = [3, 2] is 


IN}IR} oO 
|-4}/G2] © 
[nN 


The following statement is an immediate consequence of Definition 3.1 and Theo- 
rem 3.7(a): 


Ifk,n € Zso andn > 1, then forall A € Ax s,, 


dim(Z; ,) = Min = 


set-partition tableaux of shape A and content 
{O"-", 1,...,k} forsome t such that |A*| < ft <n 


(3.17) 


3.3 Bijections 


Combining the results of the previous two sections with what we know from Schur— 
Weyl duality (1.4)-(1.7), we have the following: 


Theorem 3.18 For Z;, = Ends, (M®*) and for 4 n, the following are equal: 


(a) the multiplicity mj, of Si in M&, 


(b) the dimension of the irreducible Z, ,-module rage indexed by i, 
(c) the number of paths in the Bratteli diagram B(Sy, Sy_1) from [n] at level 0 to x 


at level k, 

(d) the number of vacillating tableaux of shape x and length k, 

(e) the number of pairs (,S), where x is a set partition of {1,2,...,k} with t 
blocks, and S is a standard tableau of shape d/{n —t] for some t such that 
|A*| <t <n, 

(f) the number of set-partition tableaux of shape and content {0"', 1, ..., k}, for 


some t such that |A*| <t <n. 


The fact that (c) and (d) have the same cardinality is immediate from the definition 
of vacillating tableaux. The fact that (e) and (f) have equal cardinalities can be seen 
by taking a set-partition tableau as in (e), and replacing the entries in the boxes having 
nonzero entries with the numbers 1, 2, ..., ¢ according to their maximal entries from 
smallest to largest. The reverse process fills the boxes of the standard tableau S with 
the entries in the blocks of x according to their maximal elements with 1 for the 
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block with smallest maximal entry and proceeding to ¢ for the block with the largest 
entry. For instance, the set-partition tableau T of Example3.15 corresponds to the 
pair (77, S) given by 


m = {1,6]4,7,9,10| 2,11, 12] 8,14] 15, 16 | 5, 13, 18 | 3,17, 19 | 20}, and 


S= 7 |, 
2|6[8| 
3/4 
5 


_ 


We now describe an algorithm that gives a bijection between set-partition tableaux 
of shape A and content {0"~', 1, ... , k} forsome |A#| <t <nand vacillating tableaux 
of length k. The algorithm assumes familiarity with Schensted row insertion (see 
[37, Sect. 7.11]). We use T <— b to mean row insertion of the box b (along with its 
entries) into the set-partition tableau T governed by their maximum elements as in 
Definition 3.1. That is, do usual Schensted insertion on the maximal elements of each 
box, but then also include all the entries of the box. 


A. Set-partition Tableaux > Vacillating Tableaux 


Given a set-partition tableau T of shape 4+ n and content {0"~',1,...,k}, with 

|A"| < t <n, the following algorithm recursively produces a vacillating k-tableau 
1 

({n] = A, A@,AM,..., AW = 2) of shape A. Examples can be found in Figs. 2 

and 3. 


(1) Leta® =A, and set T = T. 
(2) For j =k,k —1,..., 1 Gn descending order), do the following: 


(a) Let T/-2) be the tableau obtained from T by removing the box b that 
contains j. At this stage, j will be the largest entry of T so this box will be 
removable. Let 4‘/~2) be the shape of T/~2), 

(b) Delete the entry j from b. If b is then empty, add 0 to it. 

(c) Let TY-) = TY-2) — b be the Schensted row insertion of b into T¥-2), 
and let AY— be the shape of TY—), 


We delete the largest number j at the jth step, so by our construction TY? is a 
tableau containing a set partition of {1,2,..., j} foreach k > j => 1. Furthermore, 
Schensted insertion keeps the rows weakly increasing and columns strictly increasing 
at each step. At the conclusion, T® is a semistandard tableau that contains only 
zeros, and as such it must have shape 1) = [n]. The sequence of underlying shapes 
(AM, a), .M,..., 4), listed in reverse order from the way they are constructed, 
is obtained by removing and adding a box at alternate steps, so it is a vacillating 
tableau of shape A and length k. 


B. Vacillating Tableaux > Set-Partition Tableaux 


Algorithm A is easily seen to be invertible. Given a vacillating tableau (A, 4), 0, 
..., A) of shape A and length k, the following algorithm produces a set-partition 
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j=45 [ofo] — [13 
j=+ [ofo[o][o]-[o 214 
j=l 0[0[0 i= |e 
i 2[4 
- 135 
1 
j=1; [o[o][o0]1]<[0 
a j= 54 0); 4 -— 0 
2 
j=2 o;o[o{l 135 
2 
j=6 0; 4 
j=25 [o]fof[o] « [1 2|6 
2 135 
j=3 0[0][0 j=6; [O;6] — [4 
2 | 13 2 
135 
j=35 [0] 0]13] - [0 
2 j=7 0 | 6 
2 (47 
135 
j=4 [0][0]13 : 
2|4 


Fig. 2 Bijection between a vacillating 7-tableau of shape [2, 2, 1] and a 7-set-partition tableau of 
shape [2, 2, 1] 


tableau T of shape 4. This process recursively fills the boxes of the shapes A‘) to 
produce the same fillings as the algorithm above. 


(1') Let T be the semistandard tableau of shape 4 = [n] with each of its boxes 
filled with 0. 
(2') For j = 0,1,...,k, do the following: 


(a’) Let T*+2) be the tableau given by un-inserting the box of T at position 
x) /.9+2), and let b be the box that is un-inserted in this process. That is, 
T+) and b are the unique tableau of shape 4+?) and box, respectively, 
such that T) = TU+) < b. 

(b’) Add j to box b. If b contains 0, delete 0 from it. 

(c’) Add the content of the box b to the box in position AV+) /A9+2), and fill 


: ' ; i441 
the rest of T/*) with the same entries as in the boxes of T!*2). 


Algorithms A and B invert one another step-by-step, since (a) and (c’), (b) and (b’), 
and (c) and (a’) are easily seen to be the inverses of one another. The fact that steps 
(a) and (c’) are inverses comes from the fact that Schensted insertion is invertible. 

Our bijection implies the following result: 
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j=0 O[o],o[o]o 
j=45 [0] 0 — [13 
2 
j=3 olololo]<|o 4 
j=l 0], o0]0 j=5 0 | 0 [135 
1 2 
4 
j=14 [ofofo][i]-[o 
j=5; [0] 2135) «Lo 
[4 
j=2 ofol[o].l 
2 
j=6 0 | 2 [135] 6 
4 
i=24 [o]o][o -[1 
2 
j=6; [0] 2 [135] <[6 
4 
J=3 0; 0/0 
2 | 13 
j=7 [0[2 [B35 @ 
4 
j=3; [olo]3] <[0 
2 
j=74 [0 | 4 [135,67] —[2 
j=4 0 | 0 | 13 
2 j=8 0 | 4 [135] 67 | 28 
4 


Fig. 3. Bijection between the set partition 2 = {4 | 135 | 67 | 28} and an 8-vacillating tableau of 
shape [5] illustrating Corollary 3.20 


Theorem 3.19 For eachiX € Axs,, there is a bijection between the set of vacillating 
tableaux of shape x and length k and the set of set-partition tableaux of shape i and 
content {0"~', 1, ..., k} for some t such that |A*| <t<n. 


Our bijection also provides a combinatorial proof of the dimension formula for 
Zx.n (see (b) of Theorem 3.7), which is illustrated with an example in Fig. 3. In this 
case, the bijection is between set partitions of [1, 2k] with at most n blocks and 
vacillating tableaux of shape [n] and length 2k. 


Corollary 3.20 For k,ne€Zs,, dim(Z;,,) = dim(Z4,,) = my, = Bk, n) = 
~. { 2k 

> é | the number of set partitions of a set of size 2k into at most n parts. 

t=1 

Proof The first equality comes from (1.7) and the second equality from the previous 
corollary (or alternatively from (1.6)). To see the third equality, we note that Msp 
equals the number of set-partition tableaux of shape [n] having content given by a 
set partition 7 € IT, with ¢ blocks for t = 1,2,...,n (the case ¢ = |[n]*| = 0 not 
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allowed). This is the number of set partitions of {1, 2,..., 2k} which have at most n 
blocks, which is exactly B(2k, n). 


Remark 3.21 We know that dim(Z;,») = dim(Z5’,,) = My, = Dycags, (Me)? 
We can see the last equality combinatorially as well. Given a vacillating tableau 
({n], [n — 1],A®,..., AP), [n — 1], [n]) of shape [n] and length 2k, let A = 1). 
Then the first portion of this tableau ({n], [7 — 1],4,...,%-2), 2) and the 
second portion ([n], [n — 1], AC, ..., A&-2), A) arranged in reverse order form 
a pair of vacillating tableaux of shape 4 and length k. This gives a bijection between 
vacillating tableaux of shape [n] and length 2k and pairs of vacillating tableaux of 
shape A and length k for some 4 € Ajs,. 


Remark 3.22 Different bijections between set partitions of [1, 2k] and vacillating 
tableaux of shape [n] and length 2k are given in [10, 18]. However, the bijections 
in those papers require that n > 2k holds. The bijection here has the advantage of 
working for all k,n € Zs. 


4 The Partition Algebra P;(n) 


4.1 The Diagram Basis of the Partition Algebra P;(n) 


Let a € Ix, be a set partition of [1, 2k] = {1,2,..., 2k}. The diagram d, of 2 has 
two rows of k vertices each, with the bottom vertices indexed by 1, 2,...,k, and 
the top vertices indexed by k + 1,k +2,..., 2k from left to right. Edges are drawn 
so that the connected components of d, are the blocks of 7. An example of a set 
partition 2 € I7\6 and its corresponding diagram d, is 


m = {1,10/ 2,31 4,9, 11|5,7| 6, 12, 15, 16/8, 14| 13} and 


9 10 11 12 13 14 15 16 
e 
a= TON 


The way the edges are drawn is immaterial; what matters is that the connected 
components of the diagram d, correspond to the blocks of the set partition 2. Thus, 
d, represents the equivalence class of all diagrams with connected components equal 
to the blocks of zr. 

Multiplication of two diagrams d,,, d,, is accomplished by placing d,, above d,,, 
identifying the vertices in the bottom row of d,, with those in the top row of d,,, 
concatenating the edges, deleting all connected components that lie entirely in the 
middle row of the joined diagrams, and multiplying by a factor of n for each such 
middle-row component. For example, if 
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then 


where 7; * 72 is the set partition obtained by the concatenation of 7, and 77 in this 
process. It is easy to confirm that the product depends only on the underlying set 
partition and is independent of the diagram chosen to represent 2. For any two set 
partitions 7), 22 © ITp,, we let (7, * 22) denote the number of blocks deleted from 
the middle of the product d,,d,,, so that the product is given by 


dx, dq, = aaa levee (4.1) 


For 2 € Zs, and for k € Zs, the partition algebra P;(n) is the F-span of {dz | 
mz € IIx} under the diagram multiplication in (4.1). Thus, dim(P;,(m)) = B(2k), 
the 2k-th Bell number. We refer to {d, | 7 € Tx} as the diagram basis. Diagram 
multiplication is easily seen to be associative with identity element 


TI a 


corresponding to the set partition {1, k+1|2,k+2|---|k, 2k}. 

Set partitions in [77,_; have k and 2k in the same block, and if 71, 72 € T1ox-1, 
then & and 2k are also in the same block of 7 * m2. Thus, for k € Z;,, we define 
P,_1(n) C Px(n) to be the F-span of {dz | w € In4-1 C Tx}. There is also an 
embedding P;(n) C P; 1 (n) given by adding a top and a bottom node to the right 
of any diagram in P;(n) and a vertical edge connecting them. Setting Po(n) = F, we 
have a tower of embeddings 


Po(n) = Pi(n) CPy(n) Cc Pi1(n) Cc Pa(n) Cc Ps1(n) Cee (4.3) 


with dim(P;(7)) = |J7x.| = B(2k) (the 2k-th Bell number) for each k € SZ. 
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Fig. 4 Hasse diagram of the partition lattice /74 in the refinement ordering 


4.2 The Orbit Basis 


For k € Zs,, the set partitions [72, of [1, 2k] form a lattice (a partially ordered set 
(poset) for which each pair has a least upper bound and a greatest lower bound) under 
the partial order given by 


ma <p if every block of z is contained in a block of p. (4.4) 


In this case, we say that z is a refinement of p and that p is a coarsening of 1, so 
that IT, is partially ordered by refinement. For example, the Hasse diagram of the 
partial order < on IT, is shown in Fig. 4. 

For each k € 5Zz1, there is a second basis {x, | 2 € IIx} of P,(n), called the 
orbit basis, defined by the following coarsening relation with respect to the diagram 
basis: 


d= yx (4.5) 


m<p 


Thus, the diagram basis element d, is the sum of all orbit basis elements x, for 
which p is coarser than z. For the remainder of the paper, we adopt the following 
convention used in [3]: 


Diagrams with white vertices indicate orbit basis elements, and 
those with black vertices indicate diagram basis elements. 


(4.6) 
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For example, the expression below writes the diagram dj)23)4 in P2(7) in terms of the 


orbit basis, 
i & oe oo eo 
8 =§ + + . + + : 


Remark 4.7 We refer to the basis {x, | a € ITox} as the orbit basis, because the 
elements in this basis act on the tensor space M®* in a natural way that corresponds 
to S,,-orbits on simple tensors (see (5.12)). Jones’ original definition of the partition 
algebra in [22] introduced the orbit basis first and defined the diagram basis later 
using the refinement relation (4.5). The multiplication rule is more easily stated in 
the diagram basis, and for that reason the diagram basis is usually the preferred basis 
when working with P;(m). However, when working with P;() for 2k > n the orbit 
basis is especially useful. For example, in Sect.5 we are able to describe the kernel 
of the action of P;,(7) on tensor space as the two-sided ideal generated by a single 
orbit basis element. 


4.3 Change of Basis 


The transition matrix determined by (4.5) between the diagram basis and the orbit 
basis is the matrix f>;, called the zeta matrix of the poset I7>,. It is unitriangular with 
respect to any extension to a linear order, and thus it is invertible, confirming that 
indeed the elements x,, 2 € IT,, form a basis of P;,(n). The inverse of f2, is the 
matrix {12% of the Mobius function of the set-partition lattice, and it satisfies 


Xn = D> Max (, p)dp, (4.8) 


<p 


where j124 (7, 2) is the (7, ~) entry of j12,. The Mobius function of the set-partition 
lattice can be readily computed using the following formula. If z < , and p consists 
of £ blocks such that the ith block of ¢ is the union of b; blocks of zr, then (see, e.g., 
[36, p. 30]), 


e 
H(t, p) = | [1G - 1). (4.9) 


i=1 


The Hasse diagram of the partition lattice of [74 is shown in Fig. 4. In the change 
of basis between the orbit basis and the diagram basis (in either direction), each basis 
element is an integer linear combination of the basis elements above or equal to it in 
the Hasse diagram. If, for example, we apply formula (4.8) to express the orbit basis 
elements x; }2)3)4 and x;;23;4 in terms of the diagram basis in P2(n), we get 
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Remark 4.11 If 2 € [x,_, C IMz,x, then k and 2k are in the same block of z. Since 
the expression for x, in the diagram basis and the expression for d,, in the orbit basis 
are sums over coarsenings of zr, they involve only set partitions in [7.,_,. Thus, the 
relations in (4.5) and (4.8) apply equally well to the algebras P,_ 1 (n). The corre- 
sponding Hasse diagram is the sublattice of the diagram for [7p of partitions greater 
than or equal to {1|2]---|k-—1|kK+1|k+2|---|2k —1|k, 2k}. For example, 
the Hasse diagram for /7; is found inside that of 7, in Fig.4 as those partitions 


(4.10) 


greater than or equal to , 


Remark 4.12 The orbit diagram I? := ie i i is not the identity element in P;(). 


To get the identity element I,, we must sum all coarsenings of I?, as in the first line 
below: 


4.4 Multiplication in the Orbit Basis 


In this section, we describe the rule for multiplying in the orbit basis of the partition 
algebra P;() using the following conventions: 
For £,m € Zso, let 


(m)e = m(m — 1)---(m—€+1). (4.13) 

Thus, (m)¢ =0 if €>m, (m)o = 1, and (m)¢ = m!/(m — £)! ifm => &. 
When z € [7Ipx, then z induces a set partition on the bottom row {1, 2,..., k} 
and a set partition on the top row {k +1,k+2,..., 2k}. If m1, m2 © Tnx, then we 


say that zr, * 7 exactly matches in the middle if the set partition that 7, induces 
on its bottom row equals the set partition that 72 induces on the top row modulo 
k. For example, if k = 4 then z; = {1,4,5| 2, 8|3, 6, 7} induces the set partition 
{1,4|2|3} on the bottom row of 7, and m2 = {1,5,8|2,6|3|4, 7} induces the 
set partition {5, 8|6|7} = {1, 4|2|3}mod4 on the top row of m2. Thus, mz; * 22 
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exactly matches in the middle. This definition is easy to see in terms of the diagrams, 
as the examples below, particularly Example 4.16, demonstrate. 

In the next result, we describe the formula for multiplying two orbit basis diagrams. 
This formula was originally stated by Halverson and Ram in unpublished notes and 
was proven in [3, Cor. 4.12]. In the product expression below, x, = 0 whenever p 
has more than n blocks. Recall that (7, * 72) is the number of blocks deleted from 
the middle row of 7 * mo. 


Theorem 4.14 Multiplication in P;(n) in terms of the orbit basis {xx}zem, is given 
by 


Yia — |P\) orem) Xp. if * 12 exactly matches in the middle, 
Xa Xn = 
0 otherwise, 
where the sum is over all coarsenings p of 1 * 12 obtained by connecting blocks 


that lie entirely in the top row of 1, to blocks that lie entirely in the bottom row of 
2. 


Example 4.15 Suppose k = 3,n > 2,andz = {1,2,3|4,5, 6} € I. Then, accor- 
ding to Theorem 4.14, 


Qa 9 
ee ao) +o) 


Example 4.16 Here k = 4,n > 5, and (z) * 22) = 2 (two blocks are removed upon 
concatenation of z, and >). 


SS. 


URS 
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4.5 Characters of Partition Algebras 


Let y, be the r-cycle (1, 2,...,7)inS, C P,(n). Forapartition 2 = [144, M2,..-, Ler] 
+ £, with uw, > O and 0 < £ <k, define yy = Yu, ® Yur @-- @ Yu, @ (0) 24 
P,(n), where “®” here stands for juxtaposing diagrams from smaller partition alge- 
bras. For example, if = [5, 3, 2, 2] and k = 15, then the diagram of y,, in P;,(”) 


wa VAL ASM MIL! 


In [17, Sect. 2.2], it is shown that characters of the partition algebra P;,(m) are 
completely determined by their values on the elements of the set {y,, | wF &, O< 
é < k}, and thus, the y,, are analogous to conjugacy class representatives in a group. 

For any integer m € Z5, let F,,(0) = F(o”), the number of fixed points of 0” 
foro € S,. For the partition 2 = [141, 2, ..., dy] F £ above, set 


Fiu(o) = Fy,(o)---Fy,(o). 


Applying the duality between S,, and P;,(m), Halverson [17, Thm. 3.22] showed that 
the character value of o x y, on M® is n'~‘F,,(c). 

The conjugacy classes of S,, are indexed by the partitions 5 | n that correspond to 
the cycle type of a permutation. The number of permutations of cycle type 6 is n!/z;5, 
where zs = 1°12 ... n* §1!8y!---6,! when 6 has 4; parts equal to i. The number 
of fixed points of a permutation depends only on its cycle type. So F,, is a class 
function, and we let F,,(5) be the value of F,, on the conjugacy class labeled by 6. 
Similarly, we let x, (6) denote the value of the irreducible S,,-character x,, 4 / n, on 
the class labeled by 6. Then, applying (1.11), we have the following (compare [17, 
Cor. 3.25]): 


Theorem 4.17 Assume n > 2k. Ford € As,, and w= [[1, f2,..-, Lr] F & with 
0 < ¢ <k, the value of the irreducible character &, for Px(n) on y, is given by 


k-£ k-£ 1 
Exe) = —— DO Fulo)xoW) = —— D0 Fulo)xno) =n! = Fuld)xn(6). 


oESn oESn dn 
Remark 4.18 In the special case that « = [1*] (the partition of k with all parts equal 


to 1), then F,,(0) = Fy (o)* = F(o)*, and Vu = |x, the identity element of P;(n). The 
result in Theorem 4.17 reduces to 


1 
dim Zj., = i(k) = | Y> Fo) x. (0), 


oS, 
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which is exactly the expression in Theorem 3.7(a) for the dimension of the irreducible 
Zx.n-module Zee indexed by A, since we can identify P,(m) with the centralizer 
algebra Z;., = Ends, (M®*) when n > 2k. 


5 The Representation %;,,, : P,(n) > Ends, (M®*) and Its 
Kernel 


Jones [22] defined an action of the partition algebra P;(m) on the tensor space Me 
that commutes with the diagonal action of the symmetric group S,, on that same space 
and showed that this action affords a representation of P;(m) onto the centralizer 
algebra Ends, (M2). In this section, we describe the action of each diagram basis 
element d, and each orbit basis element x,, of P,(m) on Me, and we use the orbit 
basis to describe the image and the kernel of this action. 


5.1 The Orbit Basis of Endg(M®*) for G a Subgroup of Sy 


Assume k,n € Zs, and let {v; | 1 < j <n} be the basis for the permutation mod- 
ule M, of S,. The elements v, = v,, @---@v,, for r=(ri,...,1k) € [1,n]}k = 
{1,2,...,n}* form a basis for the S,-module Me with S, acting diagonally, 
ON, = Vo(r) = Vo(r;) @ +++ @ Vo(ry)s a8 in (1.2). 

Suppose g = Do eecrioitt op Be End(M®&*) where {E*} is a basis of matrix units 
for End (M2), and the coefficients y? belong to F. Then Efv; = 6, 4Vs, with 6,4 being 
the Kronecker delta, and the action of g on the basis of simple tensors is given by 


(vr) = > Os. (5.1) 


For any subgroup G C S,, (in particular, for S,, itself) and for the centralizer algebra 
Endg(M®*) = {g € End(M®*) | go = o¢ for all o € G}, we have 
y € Endg(M*™) —> oy=go forall o €G 
= a Q; Vols) = a ~5(Vs forall re [1 n}*, 


se[1,n]* se[1,n]* 


and so 


yg € Endg(M™) —> gS = ge for all r,s €[1,n]*,o €G. 
(5.2) 
It is convenient to view the pair of k-tuples r,s € [1, nj|* in (5.2) as a single 2k- 
tuple (r,s) € [1, n[-*. In this notation, condition (5.2) tells us that the elements of 
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Endg(M®*) are in one-to-one correspondence with the G-orbits on [1, n]**, where 


o € Gactson (11,..., 72x) € [1,n]* by o(n,..., rx) = (6(1), «--, 6 (r2x)). 
We adopt the shorthand notation (r[r’) = (1,...,7) € U1, n* when r= 
(r1,...,7K) € [1 n}k and r = (reat, ..., rox) € [1, n]}*. Let the G-orbit of (r|r’) € 


[1, n]** be denoted by G(r|r’) = {a (r|r’) | o € G} and define 


Xin So ES (5.3) 


(sls')€G(r|r’) 


where the sum is over the distinct elements in the orbit. This is the indicator function 
of the orbit G(r|r’), and it satisfies (5.2), so X(ryr) € Endg(M®*). Let [1, n]*/G be a 
set consisting of one 2k-tuple (r|r’) for each G-orbit. Since (5.2) is a necessary and 
sufficient condition for a transformation to belong to Endg (M2), and the elements 
X rr for (r|r’) € [1, n]°*/G are linearly independent, we have the following result. 


Theorem 5.4 ForG CS, andn,k € Zs, the centralizer algebra Endg(M®*) has 
a basis {X(yv) | (r|r’) € F1, n*/G}. In particular, dim(Endg(M®*)) equals the num- 
ber of G-orbits on [1, n}**. 


When G = §,,, then since S,, acts transitively on [1, n], the S,,-orbits on [1, n]** 
correspond to set partitions of [1, 2k] into at most n blocks. In particular, if 7 € [72% 
is a set partition of [1, 2k], then 


{(1,72,---, oa) |Ta =p <> a,b are in the same block of z} (5.5) 


is the S,,-orbit corresponding to 2 (compare this with (3.4)). The condition (5.5) 
requires there to be n or fewer blocks in ; otherwise there are not enough distinct 
values rz € [1, 1] to assign to each of the blocks. 

If GCS, is a proper subgroup, then the S,-orbits may split into smaller G- 
orbits. For example, if k = 2 and G = Ay C Su, the alternating subgroup, then the 
S4-orbit corresponding to 7 = {1 | 2 | 3, 4} contains (1, 2, 3, 3) and (1, 2, 4, 4), but 
no element of A, sends (1, 2, 3, 3) to (1, 2, 4, 4), since it would have to fix 1 and 2 
and swap 3 and 4. 

When G = S,, the orbit basis has an especially nice form, since the S,,-orbits of 
[1, n]** correspond to set partitions of [1, 2k] having at most n parts. For 7 € Tnx, 
we designate a special labeling associated with z as follows. 


Definition 5.1 Let B, be the block of z containing 1, and for 1 < j < |zr|, let B; be 
the block of zr containing the smallest number not in By U Bz U-- - U Bj_;. The stan- 
dard labeling of x is (bz|b;,), where bz = (bj,..., bg) and b). = (bg41,..., bax) 
in [1, n]*, and 


bp = j if €¢€B, for € €[1, 2k]. (5.6) 


Rather than writing Xi, |p.) for the S,-orbit element determined by (b,|b/,) € 
[1, n]?*, we denote this simply as X,,. Then it follows that 
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X= DP ES = Do aE, (5.7) 
(s|s’) € Sn (bx |b,) (r[r’)eE[1,n]2* 
where 
f 1 if rg = rp if and only ifa and Dare in the same block of z, 
(Xn)! = as (5.8) 


0 otherwise. 


In this case, Theorem 5.4 specializes to the following. 


Theorem 5.9 Fork,n € Zs, Ends, (M®*) has a basis {Xx | w € [ox.n}, and there- 
fore dim(Ends, (M®*)) = B(2k, n). 


Remark 5.10 If G is any finite group and M is any permutation module for G (that 
is, g € G acts as a permutation on a distinguished basis of n elements of M), then 
G can be viewed as a subgroup of S,, and M can be regarded as the module M,. 
Thus, the method of this section applies to tensor powers of any permutation module 
for any group G. For example, an action of a group G on a finite set {x1,..., Xn} 
can be viewed as an action of the subgroup G of S, on the permutation module 
spang{x1, X2,..., Xn}. 


5.2 The Definition of ®i.n 
For k,n € Zs, define ®,, : Px(n) > End(M®*) by 


X, ifm hasn or fewer blocks, and 
Dy n (Xx) = (5.11) 


Q if has more than zn blocks. 


As {xq | © € [px} is a basis for P;,(n), we can extend ®;_,, linearly to get a transfor- 
mation, ®,., : Py(n) > End(M@*). It follows from Theorem5.9 that ®,, maps 
P;,(n) surjectively onto Ends, (M®*) for all k,n € Zs,. When n > 2k, we have 
dim(P;(n)) = B(2k) = dim(Ends, (M®*)), and thus ®,.,, is a bijection. 

From (5.11) we see that 


Besta! 1 ifr, = 1, if and only ifa and bare in the same block of z, 
nXq), = ‘ 
B 0 otherwise. 

(5.12) 
Since the diagram basis {d, | a € I;} is related to the orbit basis {x, | w € Tox} 


by the refinement relation (4.5), we have as an immediate consequence, 


®,, (d_)" = 1 ifrz = " when a and b are in the same block of z, (5.13) 
0 otherwise. 
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The map ®; ,, can be shown to be an algebra homomorphism (the diagram basis works 


especially well for doing that, see [3, Prop. 3.6]), and so ®;_, affords a representation 
of the partition algebra P;, (7). 


Example 5.14 (Example4.15 revisited) Recall that in this example az = 
{1,2,3]4,5, 6} and p = {1, 2,3, 4,5, 6 |} are elements of I7¢. Then, by definition, 
P3.n (Xx) = Diz jet ny Ei) for all n = 2 so that 
3.002) =(®3n0n))=@-2 YO BY +m-) DO et 
i~je[l,n] 7é[1,n] 
= (0 — 2)63,.n (rr) + (2 — 1)O3,n (Xp) = O3,.n((0 — 2aw + — 1xp). 


Such examples inspired the product rule in Theorem 4. 14. 


We identify S,_; with the subgroup of S,, of permutations that fix n and make 
the identification M&* = M&* @v, C M®&+), so that M®* is a submodule for both 
Sn—1 and Pyai(n) C Pxai(n). Then for tuples 7, § € [1, n]**! having ry.) =n = 
Sk41, condition (5.2) for G = S,_; becomes 


a (8) 


gs = 3@ forall 7,§¢€[1, nt 


7 OE Sn-1- 
Thus, the matrix units for G = S,_, in (5.3) correspond to set partitions in T7o,41; 
that is, set partitions of {1,2,...,2(k + 1)} having k + 1 and 2(k + 1) in the same 
block. 

Let P41, Pepi(a) > End(M®* @ v,,) be defined by 


Pinte) = D> (Xx)FEF, 


(FF )ef1 np? e+) 


where the sum is over tuples of the form r = (r1,..., 74,7), 1 = (Tha, - ++, Tok, 2) 
in [1, n]**!, and 


(X,)" = 1 iffm= " if and only ifa and b are in the same block of z, (5.15) 
0 otherwise. 


Then the argument proving [3, Prop. 3.6] can be easily adapted to show that ®,, 1 ,, 
is a representation of P,, i(n). : 

The next theorem describes a basis for the image and the kernel of ®; ,,. Part (a) 
follows from our work in Sect. 5.1 and is originally due to Jones [22]. The extension 
to P41, can be found in [19, Thm. 3.6]. 


Theorem 5.16 Assume n € Zs, and {xz | m € Ix} is the orbit basis for Px(n). 


(a) Fork € Zs, the representation By» : Py(n) > End(M®*) has 
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im Ox.» = Ends, (M&) = spanc{®g.n (Xx) | 1 € Tox has <n blocks} 
ker Dyn = Spanc {x7 | « € [2% has more than n blocks}. 


Consequently, Ends, (M2*) is isomorphic to Px(n) for n > 2k. 
(b) Fork € Zyo, the representation Dut Prat (n) > End(M®*) has 


im®,,1,, =Ends,_,(M2*) = spanc{®,,1,(%x) | t € Mak+1 has < n blocks} 


2? 
ker ®, , = spanc {xz | w € [x41 has more thann blocks}. 


n 


Consequently, Ends,_, (M2) is isomorphic to P,,1(n) forn > 2k +1. 


Remark 5.17 The assertion that the map ®;,, (resp. ®, 4 1 a) is an isomorphism 
when n > 2k (resp. when n > 2k + 1) holds because set partitions 2 € IT, (resp. 
mw € I1¢+1) have no more than n blocks under those assumptions. 


Example 5.18 When k = 2 and n = 2, the image of ®2.9 : P2(2) > Ends, (M$) 
is spanned by the images of the following 8 diagrams, 


ww llYNN AR. 


and the kernel is spanned by the following 7 diagrams, 
0000 J ° Al iw ° o—o 
°o 0’ O—O’ ro 0’ 0 0’ 0 0’0 0° 


Remark 5.19 Recall from Corollary 3.11 that B(¢,n) = (n!)7' ooes, F(o)! for 
all £2 € Zs. When n = 2, only the identity element of S» has fixed points, and we 
see that B(2k, 2) = 5(27k ) = 2**-! for allk € $Z0, in agreement with the values 
in the first column of Fig.5. When n = 3, there are three transpositions in S3, each 
having one fixed point, and two cycles of length 3 that have no fixed points. Thus, 


1 \ 37AT 1 
B(2k, 3) = =(3" +3-1%* +42. or) = —_ for k € 5Z>0. 


These values correspond to the numbers in the second column of the table. 


5.3 The Kernel of the Surjection 
@4,n : Py(n) > Ends (M®*) When 2k > n 


This section is devoted to a description of the kernel of the map ®;., (and also of 
®,_1,,) when 2k > n. Toward this purpose, the following orbit basis elements e;. », 


www.TechnicalBooksPDF.com 


36 G. Benkart and T. Halverson 


k_ [B(2k,2) B(2k,3) Bk,4) B(2k,5) BOk,6) BOk,7) Bk,8)| B(2k) 
5 1 1 1 1 1 1 1 1 
1 2 2 2 2 2 2 2 2 
14 4 5 5 5 5 5 5 5 
2 8 14 15 15 15 15 15 15 
25 16 41 51 52 52 52 52 52 
3 32. «:122,—s«187)— 202, «203, 203.—S 203) 203 
35 64 365 715 855 876 877 877) 877 


4 128 1094 = 2795 3845 4111 4139 4140 4140 
44 256 3281 11051 18002 20648 21110 21146) 21147 
5 512 9842 43947 86472 109299 115179 115929) 115975 
54 1024 29525 175275 422005 601492 665479 677359) 678570 
6 2048 88574 700075 2079475 3403127 4030523 4189550]4213597 


Fig. 5 Table of values of the restricted Bell number B(2k, 1), which equals the dimension of the 
image of the surjection Dy, : Px(n) > Ends, (M&*), The rightmost column gives dim(P;(”)) = 
B(2k), the 2kth (unrestricted) Bell number. Note that column B(2k, 5) equals the right-hand column 
of dimensions in the Bratteli diagram B(S5, S4) displayed in Fig. 1 


for k,n € Zs, and 2k > n were introduced in [3, Sect. 5.3]: 


O° fe} J in | 
apes = ifn >k >n/2, 


n+1—k 2k—n—-1 
Cn = i ; i ; a ; (5.20) 
es ifk >n. 
————— 
k 
Stn =n, if2k—1>n. (5.21) 


Observe that ifn > k > n/2, then the number of blocks in ex, is |eg,n| = 2(2 + 1 — 
k)+2k-—n—1=n+1,s0e,, is in the kernel of ®, ,. For example, 


ore) 
€416 = §5.6= 9 6 


has |es.6| = 7 blocks. The elements eg, , fork <5 andn < 9 are displayed in Fig. 6. 


Theorem 5.22 ([3, Thms. 5.6 and 5.9]) Assume k,n € Zs, and 2k > n. 
(a) The orbit basis element ex, in (5.20) is an essential idempotent such that 


(€k.n)” = Ckn€k,n» Where 


yap 1b! ifn>k>n/2 
foe ee ae 
1 ifk>n. 
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=, 2 eo oe ee oe ees ee 
o=2] [i i i i ee es es (ee 
n=3 Se GEL GRE REA «EER 
n=4 Sm 
n=s Bs GRIN GRSRI ERISIUT «UI 
n=6 Giiell ee einai ea 
n=7 ee Ge Bel 
n=8 al See 


Fig.6 The essential idempotent e, , fork <5 andn <9.Whenn < 2k, the kernel of ®, ,, equals 
the principal ideal (en) 


(b) The kernel of the representation Py, , is the ideal of P;.(n) generated by ex, when 
2k > n. 

(c) The kernel of the representation Py_1 in is the ideal of Prt (n) generated by 
Cet nn = Cn When 2k >n +1. 


Remark 5.24 For a fixed value of n, the first time the kernel is nonzero is when 
k= S(n +1) @e., when n = 2k — 1). This is the first entry in each row in the 
table in Fig. 6. For that particular value of n, (ex.2x-1)* = (- 1)*/k! €x,2%-1 by The- 
orem 5.22(a). 


The expression for e;,, in the diagram basis is given by 


een = Mae ens Pp, (5.25) 


PETIT, Te nXP 
where zz, is the set partition of [1, 2k] corresponding to e.,,. When k = $(n +1) 
and n is odd, e¢.n = 6 --o, and all p in IT occur in the expression for e;,,. When 
k = $(n+ Iandniseven, e , = ©... 4, and all p in I7>,_; occurin the expression 


for eg,n. If k > n, then e,, = z 5 B and the set partitions p € IT, that occur in the 


expression for e;,, correspond to the coarsenings of the k columns of the diagram 
Itk.n- There are Bell number B(k) such terms. In each case, the integer coefficients 
[42% (k,n, P) can be computed using (4.9). 
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Remark 5.26 The element e43 = g z z Z generates the kernel of the surjection 


43: P43) > Ends, (M®*). Below is the diagram basis expansion for e4,3 using 
the Mobius formulas in (4.8) and (4.9). The diagram basis expansion for e2,3 can be 


found in (4.10). 
cx = QRS - (EREDAR - DRT - CSTR] - GE - OT 
-THH-LID Bere ler 
DOwwewewererttssweowmeE 


There are 15 = B(4) diagram basis elements in the expression for orbit element e;,,,. 


6 The Fundamental Theorems of Invariant Theory for S,, 


Section5.2 gives the explicit construction of the algebra homomorphism ®,,, : 
P;,(n) > Ends, (M2) and shows that the partition algebra generates the tensor 
invariants of the symmetric group S,. The First Fundamental Theorem of Invari- 
ant Theory for S, says that the partition algebra generates all tensor invariants of 
S, on Ends, (M2) = (Me2k)>, as ®, , is a surjection for all k,n. A more precise 
statement is the following: 


Theorem 6.1 ((22]) (First Fundamental Theorem of Invariant Theory for S,,) For 


allk,n € Zs1, Ben: Pe(n) > Ends, (M®*) is a surjective algebra homomorphism, 
and when n > 2k, ®x., is an isomorphism, so Py(n) = Ends, (M2*) when n > 2k. 


For k € Zs, the partition algebra P;(n) has a presentation by the generators 


Sj 


r=5] 7 l ; [ ee [ l<i<k, (6.3) 
DOLEP Do] sisen 


and the relations in the next result. Additional results on presentations for partition 
algebras can be found in [11] and the references therein, and in [12] which adopts a 
Jucys—Murphy element point of view. 


ll 
o—e 
o—e 
A 
o—e 
eo—e 

io 

IA 
IA 

Pal 

| 

i 

- 
a 
i) 

— 


Ss 
II 
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Theorem 6.5 [19, Thm. 1.11] Assume k € Zs, and set p;.1 =b d<i<k-J1). 
Then P,(n) has a presentation as a unital associative algebra by generators §; (1 < 
i<xk—-l),pp @eE Zz, 1 < £ <k), and the following relations: 


(a) sp=k, 918) = 99 (i—J|>D, sisizis) =sigisisig1 (1Sisk—2); 
(b) PE = Pe, PePm =Pmpe (m ALLS), PePeripe=Pe (PL = Mk = Pepys 
(C) SiPiPi+1 = PiPit1,  SiPiS; = Pi+t, : : 
SiPj4t = Pit 5, = Pia (d<i<k-—1), 
§15141Pi41514151 = P43 (St <k—2), 


Sipe = Pes; (@#i-4,i,i+4,i+1,1+). 


Remark 6.6 It is easily seen from the relations that P;() is generated by the ele- 
ments p;, 6; = Pit, and 5; d<i<k-l1). 


Theorems 5.6 and 5.8 of [3] prove that e; , is an essential idempotent that generates 
the kernel of ®,,, as a two-sided ideal. Moreover, Theorem5.15 of [3] shows that 
the kernel of ®;_, is generated as a two-sided ideal by the embedded image e, , ® 
(1)24—” (the diagram of e,,,, with k — n vertical edges 4 juxtaposed to its right) of 
the essential idempotent e,,, in P;,(m) for all k > n. By [3, Remark 5.20], ker ®;., 
cannot be generated by ey, ® (3)®*~ for any @ such that k >n > > 5(n+ 1). 
Identifying e,,, with its image in P,(7) for k > n, we have 


Theorem 6.7 [3, Thm. 5.19] (Second Fundamental Theorem of Invariant Theory 
for S,) For all k,n € Zs\, im ®, = Ends, (M®*) is generated by the partition 
algebra generators and relations in Theorem 6.5(a)-(c) together with the one addi- 
tional relation ex, = 0 in the case that 2k > n. When k => n, the relation ex, = 0 
can be replaced with €, » = 0. 


Example 6.8 The kernel of 33 : P3(3) > Ends, (M$*) is generated by e33 = 
g z 5. and the embedded element e33 ® (24-3) principally generates the ker- 


nel for P;() for all k > 3. The image, im ®3 3 = P3(3)/ker®3 3, is generated by the 
partition algebra P3(3) with the additional dependence relation 


o- i - i - a - ee - ee - ee - 
- e+ i: ee: ee 
+2. 2 + 22d - « a. 


This dependence relation is analogous to the one that comes from setting the ker- 
nel generator <5, (—1)8"o of the surjection FS; > Endg., (V®) (V = F’) 
equal to 0 in the Second Fundamental Theorem of Invariant Theory for GLy. 


Acknowledgements The authors thank the referee for a careful proofreading and useful sugges- 
tions. 
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Affine Grassmannians and Hessenberg M®) 
Schubert Cells sieck 


Linda Chen and Julianna Tymoczko 


Abstract We give an overview of the linear algebra, geometry, and combinatorics of 
affine Grassmannians along the lines of Fulton’s Young Tableaux for classical Grass- 
mannians. We discuss geometric and linear algebraic aspects of the decomposition 
of the affine Grassmannian into affine Schubert cells in terms of coset representa- 
tives and linear models. We describe (Grassmannian) Hessenberg Schubert cells and 
show that every affine Schubert cell can be realized as a Hessenberg Schubert cell in 
a complete flag variety and as a Grassmannian Hessenberg Schubert cell in a finite 
Grassmannian. 


1 Introduction 


Let Gr, = GL, (C((t)))/GL, (C[[t]]) denote the affine Grassmannian of type A,_1, 
where C((t)) is the ring of formal Laurent series and C[[f]] is the ring of formal 
power series. It is an infinite dimensional algebraic variety that is a central object in 
algebraic combinatorics, algebraic geometry, and geometric representation theory. 
Some introductions to the affine Grassmannian in the literature include that by Lam, 
Lapointe, Morse, and Shimozono [16], which gives an overview of the Schubert 
calculus and tableaux combinatorics of affine Grassmannians, and that by Zhu [27], 
which describes connections to Kac-Moody groups, the moduli of vector bundles 
on curves, the Langlands program, and the geometric Satake equivalence. 
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The geometry of the affine Grassmannian has also been extensively studied. The 
homology and cohomology rings of Gr, were computed by Bott as a subring and 
quotient ring of the ring of symmetric functions. A basis consisting of Schubert 
varieties and opposite Schubert varieties was studied by Kostant and Kumar using 
the nilHecke ring [14], and Lam identified them as k-Schur and dual k-Schur func- 
tions, which arise also in the study of Macdonald polynomials [15]. Lapointe and 
Morse gave explicit combinatorial maps between k-Schur functions in the homol- 
ogy of the affine Grassmannian and the quantum cohomology rings of the classical 
Grassmannian [19]. 

In this paper, we give a basic and explicit linear algebraic description of the geome- 
try of affine Schubert cells, which are indexed by affine Grassmannian permutations. 
We show how this gives rise to some of the known combinatorics, as outlined in [16, 
17]. In particular, we describe several equivalent formulations of affine Grassman- 
nian permutations and their corresponding cells, for instance in terms of windows, 
skylines, the geometric linear model, the coset interpretation, cores, and partitions, 
extending analogous formulations in the case of the classical Grassmannian. 

We also study Hessenberg varieties, which form a family of subvarieties of the 
complete flag variety defined by two parameters. Here, the complete flag variety 
FI(n) is the space of flags V, = {V; C V2 © --- C V,, = C"} where each V; is an i- 
dimensional subspace. Given a nilpotent linear operator X and a nondecreasing step 
functionh : {1,2,...,n}— {1,2,...,}, the Hessenberg variety H(X, /) consists 
of the flags V, such that XV; C Vj”) for each i. 

Hessenberg varieties arise in many contexts. They were originally defined by De 
Mari, Procesi, and Shayman as a generalization of spaces that appear in numerical 
analysis [5]. When h(i) = i for all i, they are known as Springer fibers; the top- 
dimensional cohomology of Springer fibers carries a representation of the symmet- 
ric group, and all irreducible representations can be obtained bijectively by varying 
over the nilpotent conjugacy classes of X [11, 23]. When X is regular nilpotent 
(i.e., has a single Jordan block) and h(i) =i + 1 for all i <n — 1, the Hessenberg 
varieties 7{(X, h) can be used to obtain the quantum cohomology of the flag vari- 
ety [13, 22]. More generally, when X is regular nilpotent, Hessenberg varieties are 
geometrically associated with affine Grassmannians. Peterson proved an explicit iso- 
morphism between a localized quantum cohomology ring of Grassmannians and a 
localized equivariant cohomology ring of the affine Grassmannian [20]. The geom- 
etry of Hessenberg varieties is quite mysterious: we only have limited information 
about fundamental properties such as singularities of their components, even for 
Springer fibers [7, 8, 10, 12]. 

We describe Hessenberg Schubert cells which are the intersection of a Hessenberg 
variety with a Schubert cell in F/(n); their projections to a Grassmannian Gr (k, 1) 
are called Grassmannian Hessenberg Schubert cells. We show that every affine Schu- 
bert cell can be realized as a Grassmannian Hessenberg Schubert cell. We do this in 
two ways and show that for one of them, the affine Schubert cell is in fact isomor- 
phic to the original Hessenberg Schubert cell. This result does three things: (1) it 
further motivates and contextualizes ongoing work on Hessenberg Schubert calculus 
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[2, 3, 6], (2) it turns affine Schubert calculus into a finite rather than infinite problem, 
and (3) it introduces the study of Grassmannian Hessenberg varieties. 

For our purposes in this survey, we work with GL,,(C[[t]]) though SL,,(C[[t]]) 
is also common in the literature. The substantive differences between the two cases 
are slight. 


2 The Classical Grassmannian 


The Grassmannian is a central object in algebraic geometry, combinatorics, and 
representation theory. A general introduction to the geometry of Grassmannians and 
flag varieties and associated combinatorics of Young tableaux can be found in [9]. 
In this section, we review the features of the Grassmannian that are most useful to 
understand when studying the affine Grassmannian. The Grassmannian Gr(k, n) of 
k-planes in C” can be described as the quotient GL,(C)/P where P is a maximal 
parabolic subgroup of invertible block upper-triangular matrices of the form 


| 
°= (of) 
where the entries in the blocks labeled * are arbitrary (subject to the condition that 
the matrix is invertible), and the zero block is (n — k) x k. In other words, 


P ={(mj) € GL,(C): mj =Oif i> kand j <n—k}. (1) 


The Wey] group in type A,_; is the symmetric group W = S, generated by the 
simple reflections s1, 52, ..., 5,—, Subject to the relations 


s = 1 forall i 
S, Sj 41S; = $7415; S741 ifi= 1, 2, Cree | il 25 and 
SiS 57 = S75 Sj if |i = J\ >1. 


Given integers k and n such that 1 < k <n, the Grassmannian permutations 
with (possible) descent at k are the permutations o € S, such that 
a(1) <--- <a(k) and o(k+ 1) <--- < a(n). In fact, Grassmannian permuta- 
tions with descent at k are the minimal coset representatives for W/Wp, where 
Wp is the subgroup of W generated by all simple reflections except s,;, namely 
We = (sj: fj AK) SSX Sy—e. 

We now give concrete descriptions of these objects. 


2.1 Combinatorial Description of Grassmannian 
Permutations 


Grassmannian permutations have combinatorial descriptions in terms of partitions 
and bit sequences, as follows. Given a Grassmannian permutation o with (possible) 
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descent at k, we associate a partition A(o) = (a(k) —k,..., 0(1) — 1), which by 
construction lies inside a k x (n — k) rectangle. Moreover, since o is a minimal 
coset representative for its coset in W/Wp = S,,/(S, xX Sy—~), these cosets can be 
described combinatorially via a set on which W acts transitively with stabilizer (at a 
point) (s;: j Ak). 

Consider the set of n-bit sequences with exactly k zeros. For each i let the simple 
reflection s; act on an n-bit sequence by exchanging the elements in positions i and 
i+ 1. This set corresponds bijectively with the Grassmannian permutations. The 
permutation o is recovered from the bit sequence by letting the ones be indexed by 
a(1),..., a(k) and the zeros by o(k + 1),..., 0(m). We can construct the partition 
from the bit sequence as follows: starting from the northwest corner of ak x (n — k) 
rectangle, traverse the sequence by moving south for each | and east for every 0. 


Example 2.1. For Gr(4, 9), the permutation o = 236914578 is a Grassmannian 
permutation with descent at k = 4. The associated partition (oc) is (5,3, 1, 1) and 
the bit sequence is 011001001. 


2.2 Linear Model 


Linear algebra is at the heart of traditional Schubert calculus. The Grassmannian 
Gr(k, n) is often described as the collection of k-dimensional planes in C”. Given a 
k-dimensional subspace, choose k spanning vectors and write them as the columns 
of ann x k matrix. Then use Gaussian elimination on the column vectors to obtain a 
normal form for the k-plane. For instance, each element of Gr (2, 4) appears exactly 
once in the following list of matrices, where * denotes free entries in C: 


00 00 10 00 10 10 
00 10 «0 10 * 0 01 
10]°] *O]’] *O]’?] OL]? ]T OL’ ] xx 
01 01 01 * * OK * OK 


2.3. Decomposition into Schubert Cells 


Let B denote the Borel subgroup of GL,,(C) consisting of invertible n x n upper- 
triangular matrices. Then the Grassmannian Gr(k, n) has a decomposition 


Gr(k,n) = U BoP/P 
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into a disjoint union of Schubert cells Bo P/P indexed by Grassmannian permuta- 
tions o with possible descent at k. Similarly, when B™ is the opposite Borel subgroup 
consisting of invertible n x n lower-triangular matrices, there is a decomposition 


Gr(k,n)=|JB-oP/P 


into a disjoint union of opposite Schubert cells B’o P/P. 

In order to be compatible with the standard conventions for the affine Grassman- 
nian (as in the next sections), we consider the opposite Schubert cell corresponding 
to a partition A inak x (n — k) rectangle (or equivalently, a Grassmannian permu- 
tation with descent at k) [9, §9.4]. When written as in Sect. 2.2, the free entries of the 
matrices in the opposite Schubert cell corresponding to o form the transpose dual 
of , where the dual partition (n —k — Ax, ...,n — k — 1) is the complement of 
X and the transpose (or conjugate) means we take columns rather than rows of the 
partition (corresponding to our use of column vectors rather than the row vectors 
Fulton uses [9]). 

To see how a k-plane corresponds to a coset aP, we complete each n x k matrix 
in column-reduced echelon form to ann x n matrix so that the n — k rows without 
pivots in the first k columns restrict to an (n — k) x (n — k) identity matrix in the last 
n — k columns, and so that all other entries in the last n — k columns are zero. With 
this description, an opposite Schubert cell in Gr(k, n) corresponds to a collection of 
n X k matrices that are in column-reduced echelon form and that have the identity 
matrix in a fixed k x k minor. 


Example 2.2. For the permutation o = 236914578 in Gr (4, 9), the opposite Schu- 
bert cell is given below for the 4-plane. To the right we show the minimal coset 
representative for aP. Note that the free entries form the partition (3, 3,2, 2), which 
is the transpose dual of the partition (0) = (5, 3, 1, 1). 


0000 000010000 
1000 100000000 
0100 010000000 
*« *00 **0001000 
* * 00 **0000100 
0010 001000000 
* *& * O **x*k000010 
* *& * O *k*x*k000001 
0001 000100000 


3 Affine Grassmannian Permutations 


In this section, we give several descriptions of affine Grassmannian permutations 
which, analogously to Grassmannian permutations, index affine Schubert cells. We 
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use these permutations to describe various aspects of the affine Grassmannian, includ- 
ing coset descriptions and linear models. 

The group of affine permutations 5. of type Agi is the group (59, 51, 82, --++5 Sn—1) 
generated by the simple reflections so, 51, 52, ..., 5,—1 Subject to the relations 


On = 1 forall i 
SiSi+1(mod n)Si = Sit1(mod n)SiSi+1(mod n) Hf i = 0,1,2,...,n—1, and 
SjSj = S[Sj if |i -— jl >1. 


For Gr(k, n), the Grassmannian permutations with descent at k are exactly the 
elements of minimal length in their cosets in S,/(S; x S,—-;). Compare to the set of 
affine Grassmannian permutations, which are the elements of minimal length in their 
cosets in Si, /Sn- 

We now give more explicit combinatorial descriptions of the affine Grassmannian 
permutations, beginning with a parallel to bit strings in the Grassmannian case. The 
affine permutation group S, can be described as the set of bijections w : Z > Zsuch 
that 


e wit+n) = wi(i)+n foralli € Z, and 
e yw —i)=0. 
i=1 


For 0 <i <n, the simple reflections s; can be defined by 


e s(na+i) =na+i+1ands;(na+i+1)=na-+i, and 
e s; acts as the identity on all other integers. 


Because of the characterization that w(i +n) = w(i) +n, we sometimes denote an 
affine permutation w € S,, by its uniquely determined window 


[w(1),..., w(n)]. 


In window notation, the reflection s; exchanges the entries in positions i and i + 1 
for 1 <i <n—1 and wsy = [w(n) —n, w(2),..., Ww) +7]. 

Moreover, for 1 <i <n, we can divide to express wii) = nk; + o(i) with 
1 < o(i) <n. Inspecting the description of the window of w, we see that o is a 
permutation of {1,...,} and }~"_, k; = 0. It is sometimes convenient to write the 
window of w as 


[w1),...,w@] =nlk,...,k,]+[oC),...,0()]. 


We say a permutation w is an affine Grassmannian permutation if its window is 
increasing, namely w(1) <--- < w(n). This is the minimal length element in the 
coset wS, € co /Sn. Given an affine Grassmannian permutation, we reorder the k; 
according to a to define the offset sequence d(w) = (d),..., d,). In other words, for 
each i we have d,,;) = k; or equivalently d; = k,-1(. 
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Thus given an affine Grassmannian permutation w, we obtain an element d(w) 
of the set 


S= das dh) €28: Sod =o). 


i=1 


Conversely, given such an n-tuple d = (d|, dz,...,d,), we obtain a unique affine 
Grassmannian permutation whose window is the set {nd; + i} taken in increasing 
order. 


Example 3.1. Consider the permutation w = sos25\59 € Se: Its window is 
[—1,0,7], which can be written [—1,0,7] = 3[—1, —1,2]+[2,3,1] so that 
k = [-—1, —1, 2] and d(w) = (2, —1, —1). The set {nd; + i} = {7, —1, 0} taken in 
increasing order recovers the window [—1, 0, 7]. 


Example 3.2. Consider the permutation w = s2895\525\50 € Sa. Its window is 
[—4, 3,7], which can be written [—4, 3,7] = 3[—2,0,2]+[2,3,1] so that 
k = [—2, 0, 2] and d(w) = [2, —2, 0]. Then the set {nd; + i} = {7, —4, 3} taken in 
increasing order recovers the window [—4, 3, 7]. 


Ifi =1,...,n— 1thensinces;(nd; +i) = nd; +i+ lands;(ndj4; +i+1) = 
nd;+, + i, the simple reflection s; acts on an element in S by exchanging the entries 
d; and d;+, . Since so(nd; + 1) = nd; = n(d; — 1) +n and so(nd, +n) = nd, + 
n+1=n(d, + 1) + 1, the simple reflection so acts on an element d in S by sending 


(di, ..-,dn) + (dy +1, do, ...,dn—1, d) — 1). (2) 


With this identification, ce acts transitively on S and the stabilizer of (0,0, ..., 0) 
is S,, So S is in bijection with the set of affine Grassmannian permutations. 


3.1 Coset Description for the Affine Grassmannian 


We now investigate the elements of the affine Grassmannian as cosets of GL, (C[[t]]). 
By analogy with the classical Grassmannian in Sect.2, we describe the column 
echelon form as an element of GL, (C[[t]]). We use these tools later in Sect.4 to 
give a Stratification of the affine Grassmannian by affine Schubert cells, which are 
double cosets JwGL,(C[[t]]) where the Iwahori subgroup I C GL, (C[[t]]) is an 
analogue of the Borel subgroup and w is indexed by the set of affine Grassmannian 
permutations. 

For every g € GL, (C((t))), we construct a column echelon matrix by applying 
a version of Gaussian elimination on elements of GL, (C[[t]]). Since there exists a 
unique column echelon matrix in each coset gGL,, (C[[t]]), the set of column echelon 
matrices give a set of representatives for the cosets gGL,,(C[[t]]). 

Note that in linear algebra over C[[t]], one must be attentive to units. (This is 
more generally true in linear algebra over a ring R. For instance, if R is an arbitrary 
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commutative ring with identity, then an element of GL,,(R) is a matrix whose deter- 
minant is a unit in R.) In particular, an element r(t) € C[[t]] is a unit if and only if 
the constant term of r is nonzero. [1, Chapter 1, Exercise 5]. 

Let e;,..., &, denote the standard basis of C”. Given an element g € GL, (C((t))) 
with entries gj;(¢), repeat the following steps to column-reduce g: 


e Assume that the strictly upper-triangular entries of the first i rows of g are zero, 
and that the first i entries along the diagonal of g are t“, 1%, t@,..., t%. (This is 
reduced form for the first i columns and rows.) 

e Consider the entries above and on the diagonal along the (i + 1)th row of g. 
Suppose gj+1,; is the entry with the smallest minimal-degree term, and suppose 
this minimal degree is t4+1, Then Gi41,j) = t+1u(t), where u(t) is a unit in 
Ctl]. 

Replace the jth column g; of g with the vector (u (t)) !g if 


e Foreachi’ € {i+ 1,i+2,...,n} other than 7 the minimal-degree term of gj41,i 
is by definition a multiple of “+1 ¢% for some a; > 0. Hence Gi+1,i' = r+ ¢% y!(t) 
for some unit w’(t) € C[[t]]. 


Replace the (i’)th column gj; with gj — t“’u'(t)g;. 
e Exchange columns j andi + 1. 


After this step is implemented, the first i + 1 columns and rows are in reduced form, 
so the algorithm can be repeated. 

Once the previous algorithm has been repeated n times, we obtain a lower- 
triangular matrix with entries “', t”, ... , % along the diagonal, as in Example 3.3. 

We may back-eliminate in a similar way, replacing the jth column g; by an 
appropriate combination of the columns g;, gj+1,..-, Jn So that each entry gj,; is a 
Laurent polynomial with maximal-degree term t“—!. 

After this process, the column-reduced form of g € GL, (C((t))) is a matrix 


ta O --- 0 
92.1 t2 --. O 
Gn Gn.2°** tn 
where g;,; = Oifi < j, the entries 9; ; = t“ along the diagonal, and gj, ; 1s a Laurent 


polynomial with maximal-degree term ¢“—! if i > j. 


Example 3.3. We demonstrate this column reduction. Reordering according to the 
degrees of the first row transforms 


0 t it 
l+t 1421+ jst! - 
3-47 ttt 3477 42h +404 42P + 030741) 44147? 40 


to the matrix 
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¢ 0 t 
t? l+t a ee 
44 r+ 47 30447 4440 34707 +20 + 44 4 20 + 053807 4 741) 
Eliminating using the first entry of the first column gives 
rt 0 0 
tl pe t+, 04 
4414+ P40 30447440 34 55.,607% 4271) 


Reordering, rescaling, and eliminating entries in the second row of the last two 
columns gives 


t7! 0 0 t! 0 0 
tr! tr! ol+t ae ae tr! 0 
444474347 3t+47%4°4P/ 44r4+P4P34r 27 


In this case, the exponents along the diagonal are —1, —1, —2. Back-eliminating 
along the last row and then second-to-last row then gives 


t! 0 0 t' 0 0 
t! ¢lodlRedo tO 
4473 £ 14+73 ? 


By construction, the reduced form of g is in the coset gGL,,(C[[t]]). In fact, more 
is true. 


Proposition 3.4. The reduced form of the coset gGL,(C[[t]]) is unique. 


Proof. Suppose that g’ and g” are two different reduced representatives for g, namely 
that 


(a) there are matrices h’,h” € GL,(C[[t]]) with g = gh’ and g” = gh” 
(b) both g’ and g” are lower-triangular 


(c) both g! and g” have t“, t@,..., t% along the diagonal 
(d) for both g’ and g”, each lower-triangular entry (i, j) has maximal-degree term 
tu-! 


Both g’ and g” are invertible in GL,,(C((t))). The set of lower-triangular matrices 
in GL,(C((t))) forms a subgroup. In particular, the inverse (g”)~! is also lower- 
triangular, as is the product (g”)~'g’. The diagonal entries of (g”)~'g' agree with 
those of (g")~'g" so the matrix (g”)~'g' has ones along the diagonal. 

Moreover, the product (g")~!g’ = (h")~'h' is in GL, (C[[t]]). In other words, the 
product (g")~!g’ = h, where h € GL,,(C[[t]]) is a lower-triangular matrix with ones 
along the diagonal. Writing the product g’ = g’h, we conclude that the (j + 1, /) 
subdiagonal entry in g’ differs from the corresponding entry in g” by a power series 
of the form hj4i,j (tte for some h j+1,;(t) € C[[¢]]. This violates condition (d) 
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above, unless h j+1,;(t) = 0. Working backwards along subdiagonals, we conclude 
that actually h is the identity, and g’ = g” as desired. O 


3.2. Geometric Linear Model for the Affine Grassmannian 


We now extend our analogy between ordinary and affine Grassmannians to include 
the geometric linear model for Grassmannians, as certain subspaces of a vector 
space. In the ordinary Grassmannian, we obtain a k-dimensional linear subspace 
from the coset aP by constructing the span of the first k columns of a. Similarly, 
we can obtain a subspace from the coset gGL,(C[[t]]) by asking for the C[[t]]- 
linear span of the columns of g. However, in the ordinary Grassmannian, we have an 
intrinsic description of the subspaces that appear, namely the set of all k-dimensional 
linear subspaces of a fixed n-dimensional vector space. The intrinsic description of 
the subspaces for the affine Grassmannian is more complicated than for ordinary 
Grassmannians; we turn our attention to this description now. 

The next proposition characterizes subspaces in C((t)) @ C” that give elements 
of the affine Grassmannian Gr,,. Billey and Mitchell call this the Quillen model for 
the affine Grassmannian [4, page 210]. 

For g € GL, (C((t))) with columns 9g, g2,..-, Gn, consider the map 


gGL,(Cl[t]) Ly = spancyry (91, 92; +++ In} - (3) 


The columns of each element of the coset gGL,,(C[[t]]) are invertible C[[f]]-linear 
combinations of the columns of g so the map gGL,,(C[[t]]) +> L, is well-defined. 


Proposition 3.5. The map given in (3) is a bijection between the set of cosets 
gGL,(Cllt]])) € GLn(C((t)))/GLy(CI[t]) and subspaces L © C((t)) @ C" sat- 
isfying 
(a) Lisa C[t]-module, namely tL C L, and 
(b) L differs from C[{[t]] ® C" only in a finite window, namely there exists N > 0 
such that 
MW(CIec")cLer*’ (CimMec"’). 


Proof. First we prove that for every g, the subspace L, satisfies Conditions (a) and 
(b). Condition (a) holds by construction, since L, is closed under multiplication by 
elements of C[[t]] and hence in particular by t. Without loss of generality, take g 
to have the canonical form described in Sect.3.1. Thus the entries of the ith row of 
g have coefficients in C((t)) with maximum degree t“—! Let d be the minimum 
exponent that appears in any of the entries g;;(t) of g. Then by definition, each 
polynomial in each column is in 


MCh Sspane | a es} 


So each g; is in t4(C[t]) ® C” and hence L, is in t?C[[t]] @ C”. 


www.TechnicalBooksPDF.com 


Affine Grassmannians and Hessenberg Schubert Cells 53 


Similarly, let d' be the maximum exponent so that 7 appears in any of the g; j(t). 
(Note that d’ must be along the diagonal, since each g;;(t) has maximum degree less 
than the exponent of the diagonal element in its row.) The subspace L, contains the 
vectors ae. for all j = 1,2,...,2 because d’ — d > 0. Moreover, the minimal- 
degree term in each 1? ~4g; ;(t) has degree at least d’ which by construction is at least 
d;. Thus there is a linear combination of the (lower-triangular) columns g1, 92, .-.. Jn 
and an exponent di so that 


d'—d d' 
94 F Pj jr1Gjt1 H Pj,j429j42 T+ + DjnGn = 016; 
for each j, where pj,j+1,---, Pjn € C[t]. So L, contains the elements tte), t%e, 
..., te, for some exponents dj, ..., d/. In particular, L, contains t” C[[t]] @ C" 


for d” = max{d}, d,..., d/}. If N = max{|d|, d’”} then 


t% (C[[t]] @C") CL, St” (Clt]] @C"). 


Next we show that every subspace of the desired form arises as L, for some matrix 
g in GL, (C((t))). We do this by explicitly constructing the columns 91, go, ... of g 
inductively from L. Suppose that g;,..., g;-1 € L have been chosen such that every 
v € L can be written as 


veqntot-:-+9jiOgj-1 + rj Me +--+ men (4) 

where qi(f),...,qj-1(t) € C[[t]] and rj(t),...m(t) € C((a)). 
Choose vu; € L to be any vector for which the minimum exponent d; of r;(f) in 
Equation (4) is minimal. Since L C t~-"C[[t]] @ C", every power of ¢ that appears 


is at least —N and so this degree d; exists. By construction, (7; (t)t~%) € C[[t] has 
nonzero constant term and hence is a unit. Define 


gj = jt) "jy — Og — +++ — 4j-1 2) gj-1) = 4% ej; + vi 
where vi € C((t)) ® spanc {ej41, bids en}. When j = 1, this proves the base case. 


Otherwise note that since d; was chosen minimally, by adjusting Eq. (4) by a C[[#]]- 
multiple of g;, every v € L can be written 


v=eadat-:-+gjiMgj-1+9¢j;Og; +rjaiMejui +--+ +m Men 


where qi(t),...,q;(t) € C[[t]] and rj41(t), ...rn(t) € C(O). 


Continuing in this manner, we obtain 9), ..., J, where each g; has leading term 
tdi e;. Moreover, the C[[t]]-span of the g;,..., 9, is the subspace L. By construc- 
tion, the vectors gj, g2,.--,9n are lower-triangular and so linearly independent 


over C[[t]]. Thus the matrix g whose columns are the g; is in GL,(C((t))) as 
desired. 
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Remark 3.6. Note that the subspaces L satisfying Conditions (a) and (b) are in fact 
generated by Laurent polynomials since that holds for the coset representatives in 
Sect. 3.1. (This can be proven directly by back-substituting at the end of the previous 
proof, as well.) 


Example 3.7. Continuing Example 3.3, in this case d = —1 and d’ = 2. We obtain 
the three column vectors 


C07 4°70 2° .07y 


The third vector is simply t?e3 so d, = 5 in this case. We eliminate the last entry of 
t°g, and t*g2 using polynomial multiples of the original column vector (0, 0, t7)". 
This gives t?e, and te. so d, = d, = 2 in this case. (In general, we would need 
another polynomial linear combination to eliminate the second entry of t4~“q,.) 


3.3 Affine Grassmannian Permutations and Skyline 
Diagrams 


In this subsection, we give the coset description corresponding to an affine Grass- 
mannian permutation and describe its associated subspace using the geometric linear 
model of Sect. 3.2. 

For each i with 1 <i <n — 1, the affine permutations s; correspond to the usual 
permutation matrix in GL,,(C), namely the identity matrix with columns i and 
i+ 1 exchanged. The affine permutation so can also be represented as a matrix 
in GL, (C((t))) that incorporates both a classical permutation (corresponding to the 
reflection that exchanges | and n) and a rescaling by ft. Explicitly, the matrix for so 
is the monomial matrix with ones along the diagonal in rows 2,3,...,” — 1, hast 
in the top right corner, and has ¢~! in the bottom-left corner. 

Given an affine Grassmannian permutation w, we give the coset description for 
wGL,(C[[t]]). Since the ordinary permutation matrices of S,, are in GL, (C[[f]]), 
we may reorder the columns of w so that nonzero elements lie on the diagonal. This 
means that wGL,,(C[[t]]) corresponds to a diagonal matrix with rte tm 
along the diagonal, where )~"_, d; = 0. More specifically, under the map (3), a coset 
wGL,(C[[t]]) corresponds to 


L= spancyrp{t“er, te», aneaany tee} 


such that )° d; = 0. This is equivalent to the bijection between affine Grassmannian 

permutations and the set S of offset sequences d = (d,, do, ..., d,) from Sect. 3. 
The action of the affine Weyl group S, on the set S of offset sequences described 

in (2) is equivalent to the following action on C((t)) ® C” on each generator t“ep: 


e If.s; isa simple transposition withi = 1,...,n — | then s;(t“e,) = t%es,(4) where 
s; acts on the integers by exchanging 7 andi + | and fixing all other integers. 
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e The simple transposition sp acts by 
so(t“e}) = t*!e, and so(t%e,) = t**1e, for each j 


and by fixing t%e, if b A 1,n. 


One can “sketch” these elements wGL,,(C[[t]]) of the affine Grassmannian using 
Skyline diagrams. (David Nadler and Jared Anderson introduced us to this diagram- 
matic depiction; it also appears in [16, page 59] with the metaphor “above sea level,” 
though the top of our diagram is the bottom of the diagram in [16].) We create a 
bi-infinite array of dots that describes the generators for C((t)) ® C”: each dot rep- 
resents t“e, for some a, b, and each row represents t“C” for some a. Figure 1 shows 
this array for n = 3, with rows labeled by the exponent of f. 

If t“e, is contained in L then Condition (a) of Proposition 3.5 guarantees that all 
te, with a’ > a are in L; in the diagram, this means that once a single dot is in L, 
so are all dots below and in the same column. Hence we sketch L by outlining the 
generators that L contains, as shown in Fig. 1a; it is understood that L contains all of 
the dots below the line. Reading the topmost dots in each column from left to right 
gives exactly the heights dj, ..., d,. 


Fig. 1 Examples of skyline diagrams 


Condition (b) of Proposition 3.5 ensures that every LZ in Gr, can be summarized 
in a finite interval and that every L eventually contains an entire row (and hence, 
by Condition (1), all the rows beneath). In particular, a skyline diagram like Fig. 1b 
does NOT represent an element of Gr,. 


Example 3.8. The example in Fig. la represents the plane L = spancys{t7e1, 
t~'ey, t~'e3} and has skyscrapers with heights d = (2, —1, —1). This corresponds 
to the permutation So528\S9 in Example 3.1. 


Moreover, the affine Grassmannian can be partitioned according to the exponents 
of the entries t“, t@, ... , t% along the diagonal in the column-reduced form for the 
general coset gGL,(C[[t]]). This hints at the decomposition into affine Schubert 
cells, to which we turn in Sect. 4. 


www.TechnicalBooksPDF.com 


56 L. Chen and J. Tymoczko 


4 Affine Schubert Cells 


Similar to the finite case, the affine Grassmannian has a decomposition into affine 
Schubert cells; these cells can be described similarly to their finite analogues. The 
most important difference is that many entries in affine Schubert cells for the affine 
Grassmannian are not free. This has deep implications for our calculations. 

We first describe a subgroup of GL, (C[[t]]) that determines Schubert cells in the 
affine Grassmannian, analogous to the role of the Borel subgroup B of GL, (C) in 
the classical case. 


Definition 4.1. The Iwahori subgroup I © GL, (C[[t]]) consists of the elements of 
GL, (C[[t]]) that are upper-triangular mod t and invertible mod t. 


Remark 4.2. The elements of I are precisely those matrices in GL, (C[[t]]) whose 
strictly lower-triangular elements are divisible by t, and none of whose diagonal 
entries are divisible by t. 


We give two examples of matrices that are not in J, and one that is: 


(l4t-0! ( t ltt (itt -l 
No: ( 3 a) no:( 1 : ) YES: ( 2 ie 


Definition 4.3. For an affine Grassmannian permutation w, the affine Schubert cell 
Q, is the coset IwGL,(C[[t]]). 


By Sect.3.3, we may also write the Schubert cell JwGL,(C[[t]]) as It'GL, 
(C[[t]]), where (d,,..., d,) is the offset sequence d(w) and t4 denotes the diagonal 
matrix with r“, 1@,..., £% along the diagonal. 

Using Gaussian elimination, we can describe explicitly the elements of affine 
Schubert cells. In principle, we would like to conjugate each element g in t¢ by an 
appropriate permutation matrix o € S, so that the diagonal entries are reordered in 
decreasing order. However, while multiplication on the right by o~! preserves cosets 
in the affine Grassmannian (because o is in GL, (C[[t]]) for each permutation o), 
the Iwahori subgroup does not contain o unless c is the identity. 

Given an affine Grassmannian permutation w whose offset sequence is 
d(w) = (d),..., dn), let My be the set of matrices in GL,(C((t))) that satisfy 
the following four conditions: 


(i) the (j, 7) entry is ti, 
(ii) if d; < dj, the (i, j) entry is zero, 
(iii) if d; > dj, then the (i, j) entry has degree at most d; — 1, 
(iv) if the (i, 7) entry is nonzero, then its minimum exponent is at least d; or, if 
i> j,atleastd; + 1. 


Theorem 4.4. Let w be an affine Grassmannian permutation whose offset sequence 
is (d,,..., d,). Then elements of the affine Schubert cell Qy are in bijection with the 
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matrices in M,, according to the map (3) that sends g € M, to the C[[t]]-span of 
the columns of g. 


Proof. Consider the C[[t]]-span of the columns of a matrix g € M,,. This corre- 
sponds toa coset gGL,, (C[[t]]) by Proposition 3.5. Let g’ = aay . The entries of g’ 
are in C[[t]] by Condition (iv). The matrix g’ is invertible because 
det(g’) = det g - det(t*)~! = det g is aunit in C((f)) that is also an element of C[[r]]. 
Thus g’ € GL,,(C[[t]]). Condition (i) implies that g’ has ones along the diagonal and 
Condition (iv) implies that the lower-triangular entries of g’ are divisible by t. There- 
fore g' isin J andsogeé It". Hence the C[[t]]-span of the columns of a matrix in 
M, is an element of Q, = [t?GL,(C[[t]]). 

In the other direction, for any 7 = (yi;(¢)) € J, consider >tIGL,(C{[t]]). We 
wish to find a matrix g € M,, such that gGL,,(C[[t]]) = yt4GL,(C[[t])). 

By Remark 4.2 y;;(¢) is a unit in C[[f]] for all j and y;;(¢) =0 mod ¢ fori > j. 
By rescaling each column of yr4, i.e., multiplying on the right by the diagonal 
matrix with entries (yj; (t))~', there is a representative 9 E yt*GLy, (C[[t]]) whose 
jth column can be written 


ier tt +.9)_1 Mert Me; +9941 Deni t+ 9, Oen ©) 


where 9 j (t) = t% and if nonzero, the entries 9 (t) € C[[f]] have minimal degree at 
least dj when i < j andd; + 1 wheni > j. In other words, the matrix g° satisfies 
Conditions (i) and (iv). 

Choose an ordering i;,..., i, of {1,2,...,m} so that d;, < dj, <---<dj;,. We 
proceed by induction. Given a matrix oo E yt4GL, (C[[t]]) that satisfies Condition 
(i) for all diagonal entries and Conditions (ii)—-(iv) for rows i = i, ..., ig_1, we will 
describe column operations that produce a matrix g* satisfying Condition (i) for all 
diagonal entries and Conditions (ii)—(iv) for rows i = 11, ..., ix—1, ix. 

For the base case, consider row i, of the matrix g°. By minimality of d;,, we can 
eliminate all nondiagonal entries in row i, by adding multiples of the 7;th column. 
In particular, since the jth column looks like (5), we can eliminate the entry 9 pM) 
by scaling as indicated in Fig. 2. In both cases, we add a vector whose entries have 
the same minimum exponents as the corresponding nondiagonal entry in the jth 
column, so Condition (iv) remains valid. Also in both cases, the diagonal entry of the 
jth column is modified by a power series that is divisible by ¢“*! so its minimum 
entry is still r“’. Thus if needed, we may rescale the jth column by a unit in C[[t]] so 
that ¢“i is on the diagonal again. So g! satisfies Condition (i) for all diagonal entries 
and Conditions (11)—(iv) for row i = i. 

Now suppose that ge E yt"GLy (C[[t]]) satisfies Condition (i) for all rows and 
Conditions (i1)—(iv) for rows i = 11, ... ig_1. The ith column of yo can be written 


Ki, (te; pers + te Ci, + coe nig (en. 


k-1 
igs] 
in these entries by adding appropriate multiples of the i,th column of g 


The entries of row i, have the form g;; (t). We eliminate terms of degree at least dj, 


k-1 03 
, using a 
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min deg 
d; min deg min deg 
d; d; 
min deg 
d; +1 
min deg min deg 
dj+1 dl 
case i<j case i > J 
scaled by 4 column scaled by 
$4; -4i p4j—di+1 


Fig. 2 Schematic for two relative cases of ith and jth columns 


polynomial that is divisible by t¢/~“« for i, < j or that is divisible by r4+!~« for 


i, > j. By construction, Conditions (i)—(iv) hold for the resulting matrix g* when 
we restrict our attention to the i;,th row. 

Note that Condition (ii) for i = i,,...,i,_, for the matrix gf ' implies that 
the entries gi) (t) = +--+ = gf), (1) = Osinced;, < +++ < di,-1 < d;,. Thus adding 
C[[t]]-multiples of column i, leaves the entries in rows i),...,i,—, unchanged. 
Therefore, Conditions (1)—(iv) hold for the matrix gf fori =i,,...,i,_ 1 as well. 

Finally, by the schematic in Fig.2 and the same argument as for g!, the diagonal 
entry in the jth column for j > i, can only be changed by terms of degree strictly 
greater than d;, so we may rescale the jth column for j > i, by a unit to preserve 
Condition (i). Thus Condition (i) holds for the matrix gi for all rows. 

Taking g = g”, this completes the proof. 


In this geometric description of affine Schubert cells, if we set all nondiagonal 
entries of the matrix g to zero, we obtain the plane 


L= spaney{t“er, t%e, asd tme,} 


corresponding to a skyline diagram. This is consistent with the geometric analogy 
between affine and ordinary Schubert cells: each is described as the span of a collec- 
tion of vectors parametrized freely by variables; setting those variables to zero gives 
the unique Weyl group element in the cell. 


Example 4.5. Consider the case when the offset sequence is (—1, —1, 2) and y is 
1+t 1 0 


t 1 0 
t+5P4+P 44+ 1-¢ 
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The matrix g° is the product 


mt4+1 ¢! 0 Viso(-0! 00 rt an) 
1 rn) 0 10 =| Diso0! r! 0 
145t+7 4442-27) \O 0 Yisot! (1+ 5t+P)(Sjs0(—!) 442? 


Using the original ordering of the columns, we obtain g' by scaling the middle 
column of the following by 1 + t: 


pt 0 . 0 
Diso(—t)' f= 2 gt) 0 
+ 5+ 2)(Cjeo(—) 441-1 t 5t + P(X js9(—9!) P 


In other words 


a 0 0 
= yea(—t) = 0 
(1+5t+2)(Yjs9(-') 44+0040)-4+5t4+2) 2 


In fact, the (3, 2) entry simplifies to 3. To get g? we subtract t)iso(—1)! times the 
second column from the first: 


¢1 0 O 
g =|0 ee) 
(1+ 2tt+P)(Y (9) 3 2? 


Finally, we eliminate all terms of degree at least 2 from the third entry of the first 
column to get g°: 


t' 0 0 
g=|0 t'0 
1413 ¢ 


In other words, this continues Example 3.3. 


5 Descriptions of Affine Schubert Cells: Bit Sequence, 
Linear Model, Cores, and Partitions 


This section defines the combinatorial concepts of the core and the partition associ- 
ated to an affine Grassmannian permutation w and relates these to the linear algebraic 
model of the previous section. 

Much of the combinatorics of the (co)homology of affine Grassmannians is formu- 
lated in terms of cores and partitions. We first describe a bi-infinite bit sequence p(w) 
associated to an affine Grassmannian permutation w and called the edge sequence of 
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w [16]. The edge sequence p(w) = ... p_1 popip...is given by setting Prag+uii) = 0 
for 1 <i < nandd > Oandall other p; = 1. We often write p = ... p_i|popip... 
inserting the symbol | to form a line between p_, and po. 

The diagram of the core is constructed by tracing from northwest to southeast, with 
a south step for every one and an east step for every zero in p(w); in this sense, it is 
like a Young diagram. For instance, the edge sequence p(id) = ... 11111|10000. 
corresponds to the empty diagram. The core can also be sonstaicted using the natural 
action of a on the bit sequences p(w) induced from the association between s. and 
functions Z — Z. In other words, applying the reflections of a reduced word of w 
from right to left to p(id) yields the bit sequence p(w) as well as its associated core. 

Let ~ be the total order on the generators of C((t)) ® C” 


{t"e,:a€Z,1<b<n} 


defined by 


atl, 


a-l6) < te, ~ te) < +++ < te, <t Kee, 


-<t 


When we rewrite the results of the previous section in terms of this total order, the 
Gaussian elimination used is essentially ordinary Gaussian elimination. We will see 
that the bit sequence p(w) is encoded in the linear algebraic model. Moreover, the 
reindexing of the generators lets us read the associated core and partition almost 
immediately. Section 6 uses ideas similar to these in the context of the flag variety. 

More precisely, we write the generators {t“e,} as the totally ordered set { f;} jez 
reindexed via the correspondence fia+p <> te, for all a € Z and 1 < b <n. This 
bijection satisfies the following: 


e the total order on the generators {t“e,} of C((t)) ® C” is equivalent to the total 

order on the { f;}, in the sense that te, < t” ey if and only ifna+b <na'+b’, and 
e the C[t]-module structure on C((t)) ® C” gives a C[t]-module structure defined 

byt: fi = fitn 

With this notation and reinterpreting Conditions (i)—(iv) in terms of the total order 
~<, we can rewrite Theorem 4.4 as follows. We describe the generators of subspaces 
using two different bases, though the identification between the bases is so natural 
as to sometimes make this distinction confusing. We also describe two different 
generators, the second of which eliminates some unnecessary repetition. 


Corollary 5.1. Let w be an affine Grassmannian permutation w whose offset 
sequence is (d\,..., d,). Then elements of the affine Schubert cell Q, are in bijection 
with the collection of subspaces L that are the C[[t]]-span of 


e the vectors v1, ..., Un given by 
vj=tte; + > Baste 
= J j,a,b b 


where the sum is over t4i ej < te, anda <d+b, or 
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e the vectors v;,..., v), given by 


vi, = Sfnaj+j + Oj k Sk: 


ndj+j<k,pr=1 


Proof. First we confirm that the four conditions for M,, from Theorem 4.4 are 
equivalent to the conditions on v;. The requirements that both 1% e; < te, and 
a < dy for 3;,a,» to be nonzero mean that t4i is the only nonzero coefficient of e; in 
v;. This is Condition (i). Conditions (ii) and (iv) are captured by t4i e; < te, and 
Condition (iii) by a < dp. 

We find the {vi} by rewriting {v;} in terms of the generators { f;, } and then adding 
a convenient normalizing condition. If f; corresponds to te, then the condition 
ttie j < t%ep corresponds to nd; + j < k. Thus we need only determine a condi- 
tion equivalent to a < d,. The affine Schubert cell is the C[[t]]-span of the vectors 
VU], «++, Un SO it is also generated by the larger set of vectors ¢@ - v ; with d > 0 and 
1 < j <n. The leading term of the generator ¢@ - v jj is othe ; and the set of yore ej 
corresponds exactly to those f; with py = 0, so we will consider only generators 
with zero in those entries. This gives the second condition. In other words, write the 
matrix (vj, U2,...,fU),fU2,..., fv}, t7v2,...) in terms of the generators f; and 
then use “column operations” to obtain a reduced echelon form with columns 


vi = fnaj+j + a j,k fk 


ndj+j<k,pr=1 
and 
d / 
+0; = fnd+(ndj+j) + y Qj.k—nd Sk 


nd+(ndj+j)<k, pp=1 


for each j with 1 < j <n and each d > 0. (Those “column operations” are in 


fact equivalent to forming a linear combination of the v,, v2,..., v, over C[[t]] 
so this process is algebraically valid.) The C[[t]]-span of the vj, v5,..., uj, is the 
same as the C[[r]]-span of the set vj, v},..., fv}, fv, ..., fv}, Pv) ..., proving the 
claim. 


The shape formed by the variables in the reduced echelon form for the matrix 
[vj, U2,..., fv), fv2,...] 


is exactly the core. The independent free variables are precisely a; , with 1 < j <n 
and p, = 1; these appear exactly once inv}, v5,..., v). Thus restricting the reduced 
echelon form for (v;, v2,..., fV,, fU2, ...) tothe columns associated to vj, V2, ..., Up 
gives an associated partition, which is clearly (n — 1)-bounded since no row has free 
entries in more than n — | columns. 

The following examples demonstrate this correspondence. In our examples, we 
write the reduced vectors only until we reach that particular reduced vector t¢ vi for 
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which all generators that come after the pivot of r“v’, are contained in the affine 
Schubert cell. We also use the order < on the vectors’ leading terms to obtain a 
lower-triangular matrix; each column sums to one of vj, v5,..., fv}, fu5,.... 


Example 5.2. For w = so with window [0, 2, 4], the affine Schubert cell in GL3 
(C((t))) corresponds to te,, eo, t~'e3 and bit sequence p(w) = ... 11|0100000.... 
It is given by the C[[t]]-span of v, = te), v2 = €2, 3 = t~'e3 + ae}, or equivalently 
by the C[[t]]-span of vi = fa, vs = fs, v4 = fo +afi, tvs. The reduced echelon 
form for the range 0 < k < 4 (bolded in the above bit sequence) is 


t~!e; 1000 fo 
e, | a000] f 
e 10100] f 
a booie | - 
te, \0001) f, 


where the columns give the generators v3, V2, tv3 — AV,, V1, respectively. (We think 
of the third column as associated to tv3 even though it is in fact a linear combination 
with leading term the same as that of tv3.) In this case, the core and partition coincide 
and are both (1), representing the unique (and free) variable. 


Example 5.3. For the window w = [—1,0,7], p-1 = po = ps =-::-=0, po= 
P3 = Po = ++: = 0, and p7 = pio = --- = 9, So that p(w) = ...11110|0100100000 
.... The reduced echelon form has all zeros for k < —1, and the range -1<k <7 
whose p-values are in bold above, is pictured below. 


vy, wy th th CUT 
tlesl 0 0 0 0 0 0 fy 100 
tle} 0 1 0 0 0 0 Ol fo 010 
e: }ar102., 0 0 0 0 OF * *0 
a!|o0 01 0 0 00/lf 000 
eg | 0 0 0 1 0 0 Of 000 
te; | ai4a24ai102.; 0 O Of fa * * 0 
ta} 0 0 0 0 1 =O Offs 000 
ix | 0 0 0 0 0 1 O1F 000 
fe \0 0 000 0VG 001 


The core (4, 2) comes from all of the nonzero variables in this cell. The free variables 
are indexed by restricting our attention to the columns corresponding to v}, v4, and v| 
as sketched in the matrix on the right. In this case, the associated bounded partition 
is (2, 2). (Examples 3.3 and 4.5 both concerned an element in this cell.) 


Example 5.4. For the window w = [—4,3,7], p-4 = p-1 = po = ps =::: =0, 
P3 = Po = ++: = 0, and p7 = Pio = ++: = 0, Sothat p(w) = ... 11110110|11001000000 
... The associated core is (4, 2,2, 1, 1). 
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In sum, affine Grassmannians have affine Schubert cells that can be described 
linear algebraically by analogy with the Schubert cells in the finite Grassmannian. 
The finite Schubert cells are indexed by either of two sets of combinatorial objects: n- 
bit sequences with zeros (corresponding to the basis vectors contained in the Schubert 
cell) or Young diagrams (corresponding to the shape created by the free entries in the 
Schubert cell). Similarly, the affine Schubert cells are indexed by either: sequences 
(d\,...,d,) € Z" with }¢ d; = 0 (corresponding to the exponents along the diagonal 
of the affine Grassmannian permutation) or Young diagrams (corresponding to the 
entries in the Schubert cell parametrized by variables). Here, the dimension of an 
affine Schubert cell ©, is equal to the length @(w), which is also equal to the number 
of boxes in the associated bounded partition. 


6 (Grassmannian) Hessenberg Schubert Cells and Varieties 


In this section, we review several key facts and definitions about Hessenberg varieties. 
We use notation and terminology from type A, though many of these facts extend to 
general Lie type. We define Hessenberg Schubert cells in the flag variety and their 
images when projected to a Grassmannian, which we call Grassmannian Hessenberg 
Schubert cells. We show that a class of Hessenberg Schubert cells are isomorphic to 
affine space, a fact that we use in Sect. 7. Finally we give conditions for the image 
of the closure of a Hessenberg Schubert cell to be the closure of the corresponding 
Grassmannian Hessenberg Schubert cell. 


6.1 Hessenberg Varieties 


Let V. = V; C V2 C--- C V, = C” beacomplete flag in C”. The space of such flags 
forms the complete flag variety F/(n) which can also be described as GL,,(C)/B 
where B is the Borel subgroup of invertible n x n upper-triangular matrices. 

In this and the next section, we use the upper-triangular Borel to describe the flag 
variety. This leads to different conventions for combinatorics and linear algebra than 
in the previous sections. Reconciling these notations is the key point of the results 
in the last section. 


Definition 6.1. Fix a linear operator X :C" — C". Fix a nondecreasing function 
h:{1,2,...,n} > {1,2,...,n} with h@) = i for alli. The Hessenberg variety of 
X and h is denoted H(X, h) and defined to be 

H(X,h) = {Ve : XVi © Vii}. 


Here, the function h is called a Hessenberg function. 
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Equivalently, we can describe Hessenberg varieties using Lie theory. Write M,, 
for the set of n x n complex matrices and 6 for the set of upper-triangular matrices 
in M,,. A Hessenberg space H is a linear subspace of M,, that satisfies the following 
two properties: 


(1) H contains the upper-triangular matrices b, and 
(2) H isclosed under Lie bracket with the upper-triangular matrices, so[b, H] C H. 


Since Hessenberg spaces are closed under Lie bracket with the diagonal matrices, 
each Hessenberg space is a sum of root spaces. Hence, each Hessenberg space can 
be denoted by the subset of roots Ry satisfying 


H = (Ei :i=1,2,....n)8 GD (Ei) 
G,JERu 


where £;; denotes the matrix with | in the (i, 7) entry and zero elsewhere. 
Given a linear operator X : C” — C” anda Hessenberg space H, the Hessenberg 
variety of X and H is 


H(X, H) = {mB € GL,(C)/B:m~'Xm € H}. 


(Since H is closed under Lie bracket with 6 it is also closed under conjugation by 
elements of B, so Hessenberg varieties are in fact well-defined.) The two definitions 
of Hessenberg varieties are equivalent because there is a natural bijection between 
Hessenberg functions and Hessenberg spaces that is given by 


AH = (Ej; :i <h(j) forall j =1,2,...,n). 


Hessenberg varieties have a natural partial order induced by inclusion of vector 
spaces in M,,. This is equivalent to the partial order on Hessenberg functions given 
by 

h > fh’ if and only if h(i) > h'(i) (6) 


for alli = 1,2,...,n. 

In this paper, we assume X is nilpotent. In this case, the Jordan canonical form 
of X determines a Young diagram by the rule that if the Jordan form has blocks 
of dimension d,; > dz > --- then the Young diagram has rows d; > d2 > ---. Our 
convention is to use Young diagrams that are left-aligned and top-aligned. We now 
define a particular permutation using the Jordan form of X (Fig. 3). 


Definition 6.2. Fill the boxes in the Young diagram for X with the numbers 
{1,2,...,} without repetition, starting at the lower left corner and moving up 
the first column, then up the second column, and so on. The permuted Jordan form 
of X is the matrix X = > Ej; where the sum is taken over pairs |i\j| in the filled 
Young diagram. 
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3/916 
2/4 
i & X = Hos + E35 + E56 


Fig. 3) An example of permuted Jordan form 


6.2 Hessenberg Schubert Cells and Varieties 


Hessenberg Schubert cells are obtained by intersecting a Hessenberg variety with a 
Schubert cell in F/(m). Our convention is to associate a permutation o € S, with the 
permutation matrix (also denoted o) so that for each 7, the ith row of the matrix a 
is the basis vector e,;). The permutation matrix for 7~! is transpose to that for 7 so 
that the permutation o~! can be read from the columns of c. 

The complete flag variety F'/(n) has a decomposition 


Fl(n) = U BoB/B 


into a disjoint union of Schubert cells Cz := BoB/B indexed by o € Sy. 


Definition 6.3. Let H(X,h) C Fl(n) be a Hessenberg variety. The Hessenberg 
Schubert cell for H(X, h) anda € Sy, is the intersection Cy 7 x,n) = H(X,h)N Cz. 
The corresponding Hessenberg Schubert variety is the closure 
Con (x,n) in H(X, h). When X and h are understood, we write Cy, and Cyn. 


A Hessenberg Schubert cell can be a complicated topological space. For instance, 
Kostant and Rietsch studied an important paving of Peterson varieties created from 
nonaffine Hessenberg Schubert cells [13, 22]. For our purposes, we want Hessenberg 
Schubert cells to be affine cells. This is true under the conventions of Proposition 
6.4, when X is a nilpotent matrix in permuted Jordan form and the Schubert cells 
are written with respect to the upper-triangular Schubert decomposition. 

The next proposition shows that when X is in permuted Jordan form, each Hessen- 
berg Schubert cell in the flag variety is isomorphic to an affine space and constructs 
that isomorphism. The proof is not new; the maps identified by the second author in 
earlier work [24, Lemma 5.2 and Theorem 6.1] are actually isomorphisms, though 
the earlier paper only used the fact that the maps were homeomorphisms. 


Proposition 6.4. Suppose H(X, h) is a Hessenberg variety for a nilpotent operator 
X in permuted Jordan form and a Hessenberg function h. Fora € S,, the Hessenberg 
Schubert cell C,,44 is nonempty if and only if the flag cB € H(X, h), and in this case, 
C,.1 is isomorphic to an affine space C‘ for some ¢ > 0. 


Proof. The fact that the intersection H(X, h) MC, is nonempty if and only if the 
flag 0B € H(X, h) was proved in the original result [24, Theorem 6.1]. 
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We sketch the argument of the isomorphism to affine space, referring to details in 
[24] as needed, and checking at each stage that the homeomorphisms in that work 
are in fact isomorphisms. 

In (24, Theorem 6.1], Tymoczko showed that the Hessenberg Schubert cell Co, 
can be described as an iterated tower of affine fiber bundles 


73 Tn-2 


> L3 —> --- —> Zy-1 


where Z; is homeomorphic to the Hessenberg Schubert cell itself, the base space 
Zn—1 18 homeomorphic to affine space C&-! and each map 7; : Z; —>Zj41 is a 
fiber bundle whose fiber is an affine space C“. Explicitly, each Z; is defined as 
a subset of the unipotent group U of upper-triangular matrices with ones along the 
diagonal, as follows. The group U can be factored uniquely as U = U,_,U,_2--+ U, 
where each U; is the subset of U whose nondiagonal entries all lie on the ith row. 
The maps 7; : Z; —> Z;+1 send the element u,_,Uyj—2---Uuj+1Uu; € Z; to the element 
Un—|Un—2°**Uji44 © Zit. 

In other words, each Z; is isomorphic (as an algebraic variety not a group) to 
a product space, and each map 7; is the algebraic map given by projection to one 
factor. 

Reference [24, Theorem 6.1] also defined the homeomorphism between Z, and 
the Hessenberg Schubert cell C,,7, as multiplication T +> To composed with the 
quotient map G + G/B that sends the matrix To to the flag To B. 

These maps are both algebraic. When we restrict the inverse map to a single 
Schubert cell, the inverse map from G/B to G is also algebraic. So in fact Z, is 
isomorphic as a variety to C,,7. 

Reference [24, Lemma 5.2] describes the fiber bundle structure of each Z; as 
follows. Given u’ € Z;+; the fiber a (ul ) is the set of solutions x, to a system 
of equations x, M,, = v,. The system is affine linear, meaning that each equation 
is polynomial of degree one and may have a constant term. Both M,, and v,, vary 
continuously in u’ by conjugation. Reference [24, Lemma 5.2] identified a fixed set 
of indices J so that the entries in positions J of the matrices in 7; ' (u’) are free and 
showed that the map sending u’u +> (u’, (Ujx)xer) is a homeomorphism with inverse 
determined by the system x, My = vy. 

This map is algebraic because it realizes the varieties as product spaces and then 
projects to certain factors. The inverse map is also algebraic because it is the solution 
to an affine linear system. Thus each Z; is isomorphic as a variety to the product of 
Zi41 with the affine fiber C“. 

Finally the space Z,,_; is affine itself because it is a subgroup of U,,-; in which 
certain entries, determined by a, are set to zero [24, Definition 2.3 and Theorem 6.1]. 
This shows that the Hessenberg Schubert cell is in fact isomorphic to affine space, 
of the same dimension as in the original result. 


www.TechnicalBooksPDF.com 


Affine Grassmannians and Hessenberg Schubert Cells 67 


6.3 Grassmannian Hessenberg Schubert Cells 


For a fixed Hessenberg variety H(X,h) C Fi(n) and an integer 1 <k <n, we 
describe Grassmannian Hessenberg Schubert cells as projections of Hessenberg 
Schubert cells to the Grassmannian Gr (k, 1). 

There is a natural projection map from the complete flag variety to the Grassman- 
nian given by 7; : Fl(n) — Gr(k, n) that sends the flag V, to its k-dimensional part 
V,.. If Gr(k, n) is realized as a quotient G/P;, by a maximal parabolic P, then the 
projection may be written as the quotient map 7% : G/B — G/Py. 


Definition 6.5. Given k and a Hessenberg Schubert cell Cy... the Grassmannian 
Hessenberg Schubert cell is the image 


Tk (Cov) 
in Gr(k,n). The corresponding Grassmannian Hessenberg Schubert variety is the 
closure Tr (C..2) in Gr(k, n). 
The next lemma follows directly from the corresponding fact for flag varieties. 


Lemma 6.6. The image of the projection 7, : Co,44 > Gr(k, n) is nonempty if and 
only if the collection {Vx : Ve € Co,4} contains the k-plane (e,-1(1), @g-1@), +++ Co-1(k)) 


Proof. Proposition 6.4 stated that C,,7, is nonempty if and only if oB € H(X, h). If 
B consists of the upper-triangular matrices, then the k-dimensional part of the flag 
OB is (€,-1(1), €g-12), +++» @o-1(k)) by definition. The set 


{Vi :We Con} 


is the image of C4, under the projection to Gr(k, n), so the claim follows. 


6.4 Hessenberg Schubert Varieties and Grassmannian 
Hessenberg Schubert Varieties 


We now give a dimension condition for the image of a Hessenberg Schubert variety 
under the projection map to be a Grassmannian Hessenberg Schubert variety. More 
precisely we have the following. 


Proposition 6.7. Let} := H(X, h) be a Hessenberg variety in F1(n) for anilpotent 
X in permuted Jordan form and a Hessenberg function h. For 1 <k <nanda € Sn, 
if the dimension of the Hessenberg Schubert cell Cz,74 is equal to the dimension of 
the Grassmannian Hessenberg Schubert cell ™(C,,71) then 


TK(Co,H) = Te(Co,71)- 
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Proof. The image of a closed irreducible topological space under a closed continuous 
map is irreducible (and closed). The closed set C,,7, is irreducible because C,.7 is 
isomorphic to affine space, as proven in Proposition 6.4. Thus the image 7, (C,,7) is 
a closed irreducible set, and since it contains the Grassmannian Hessenberg Schubert 
cell m(Co.74)s 


TK(Co,H) & TK(Co,71)- (7) 


This gives dim 7;(C,.74) < dim 7, (Con) < dim Cy hic The dimension of each open 
set is the same as the dimension of its closure, so by our hypothesis on the dimensions, 
these inequalities are all in fact equalities. Since 7% (C,,71) is closed and irreducible, 
Equation (7) becomes an equality as needed. 


The dimension of a Hessenberg Schubert cell is usually larger than that of a 
corresponding Grassmannian Hessenberg Schubert cell. However, there are many 
natural examples when they are the same, e.g., Sect. 7. The previous proposition 
suggests that when equality holds, the Schubert calculus of Hessenberg varieties 
could be computed through Schubert calculus in the Grassmannian. Results on the 
dimension of Hessenberg Schubert cells can be found in [21, 26]. 


6.5 Minimal Hessenberg Schubert Cells 


Before applying the results of Sect.6 to affine Schubert cells in the next section, 
we consider minimal Hessenberg Schubert cells for fixed X and w and prove that 
they are well-defined and unique when X is nilpotent. Recall the partial order on 
Hessenberg functions h’ < h given by (6). 


Definition 6.8. Fix an n x n matrix X and a permutation o € S,. A Hessenberg 
function h is a minimal Hessenberg function for X and o if 


© Conx,n) # Bin Fl(n) and 
e for every Hessenberg function h' < h the Hessenberg Schubert cell Cy,41(x,n) ts 
empty. 


In this case we call Cy,74(x,n) 4 minimal Hessenberg Schubert cell for X and o. 


The next result shows that minimal Hessenberg Schubert cells are well-defined 
and unique. 


Proposition 6.9. Let X be a nilpotent linear operator in permuted Jordan form. The 
minimal Hessenberg Schubert cell for X and o is well-defined. In particular, there 
is a unique minimal Hessenberg function h with Cy r1x,n) # ¥. 


Proof. From Proposition 6.4, the Hessenberg Schubert cell Cg, 44x.) is nonempty if 
and only if the flagoB € H(X, h).The flagoB € H(X, h)ifandonlyifo~-'Xo ¢ H 
by definition. 
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Fig.4 An example of shadows for 0 = (14)(325) and X = E)3 + E24 + E45 (shaded), with min- 
imal Hessenberg function h = 33345 (Hessenberg space outlined) 


Each Hessenberg space H is a direct sum of root spaces because it is closed under 
the adjoint action of t. Hence if a Hessenberg space H contains an element )° ¢;; Ej; 
then it contains every root space (E;;) for which c;; 4 0. If o~'Xo € H then H 
contains (E,(i),.(j)) for each pair i, j that appears in the sum X = a Ejj. 

In particular, define a Hessenberg space by 


H =6+ QEij i < oi’) and j > o(/’)) 


where the sum is over all pairs (a(i’), o(j’)) that appear in o~'Xo = ye Eo(i),o(j): 
This is a Hessenberg space by construction. It contains o~! Xo also by construction. 
Conversely, any Hessenberg space containing o~'!Xo must contain H. 

The Hessenberg function corresponding to this Hessenberg space H is defined by 


i 
h(i) = max ? h(i — 1) 
o(j,) where E ;,, ;, is in permuted Jordan form for X and j2 = ao! (i). 


Recall that Fj, ;, is in the permuted Jordan form for X if and only if | j;| j2| 1s in 
the Young tableau describing permuted Jordan form (Fig. 4). 

The minimal H is easy to describe schematically. If Ej), ; is aroot vector, define the 
shadow of Ej; to be the span (£;; : i’ < i and j’ > j). Given an element X € M,, 
with X = > ci; E;;, define the shadow of X to be the sum of the shadows of E;; 
for which c;; # 0. Given X and o, denote the shadow of o~'Xo by V. Then the 
minimal Hessenberg Schubert cell for X and o is defined using the Hessenberg 
space H =V-+6. 


7 Affine Schubert Cells and Hessenberg Schubert Cells 


In this section, we show that every affine Schubert cell Q2,, is isomorphic to a Hes- 
senberg Schubert cell. Denote by h;, the Hessenberg function defined by h;g(i) = i 
for all 7. 


www.TechnicalBooksPDF.com 


70 L. Chen and J. Tymoczko 


Theorem 7.1. Every affine Schubert cell is isomorphic to a Hessenberg Schubert 
cell. More precisely, given an affine Grassmannian permutation w, the corresponding 
affine Schubert cell Q2,, is isomorphic to a Hessenberg Schubert cell Cy,34(x,n,) for 
some nilpotent linear operator X : CN — CN and permutation o € Sy. 


In order to prove the theorem, we first give two constructions identifying Q,, with 
a Grassmannian Hessenberg Schubert cell in a finite Grassmannian. 


Proposition 7.2. Given an affine Grassmannian permutation w with corresponding 
affine Schubert cell QQ, there are integers 1 < K < N, a projection map 7 : Gry > 
Gr(K, N), and a nilpotent linear operator X : CN + C% so that: 


(a) there is a Hessenberg function h with corresponding Hessenberg variety 
Hn := H(X, h) and a permutation T € Sy for which the Grassmannian Hes- 
senberg Schubert cell is the projection of the affine Schubert cell to Gr(K, N): 


TK(Cr.14,) = T(Qw) 


(b) there is a Grassmannian permutation T € Sy so that TB € Hia := H(X, hia) 
and 
TK (Ce Ha) = T(Qy) 


Proof. Given an affine Grassmannian permutation w and its corresponding affine 
Schubert cell Q,, we start by defining N, K, and X. 


e A positive integer N: Let t“ ey be the lowest-ordered generator with respect to 
< that is not contained in every plane in Q,,. Let t%e, be the highest-indexed 
generator that appears as a pivot in Q,,. Define a” to be a’ ifb — 1 > Db’ anda’ + 1 
otherwise. Let N = n(a” — a) and consider the vector space V spanned by the N 
vectors 

at+l 


at+l atle, Baipeed (tele, pit26 


(t7 ep, t7 epi, t%ep42,.--, t*én,t 


ey,t * e2,t Lewsey 


N vectors 


The vector space V of dimension N is called the slice of the affine Grassmannian. 
(In principle, we want the slice to be the C-span of the vectors from fe, to t+! ey 
inclusive, so that each vector e1, é€2,..., @, iS represented in the slice. However, 
we expand the slice for notational convenience so that N is divisible by n.) 

e A positive integer K < N: Write the vectors spanning a generic plane in the affine 
Grassmannian cell as 


vj, tvy,..., fv, 
U2, tU2,...,f@ v2 
Un Un... 5 tp 


where each a; > 0 is defined by the condition that the leading term of t“v; is in 
the slice but r“*!v; is not. Assume without loss of generality that the vectors are 
indexed satisfying the constraint that 
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leading term of v; > leading term of v2 > --- > leading term of v,. 


Let K be the number of nonzero vectors in this set. 

e We identify each vector in the affine Grassmannian with its image under the pro- 
jection x : Gr, > Gr(K, N) inside NV. We conflate terminology slightly and say 
that this is the image of Gr, inside the slice. 

e We reorder the nonzero vectors vj, tuj,..., f' v1, V2, tu2,...,f@ v2,..., Un, tUn, 
..., fv, in decreasing order of leading terms and call these reordered vectors 
Uj, U2,...,UK. (Note that wu; = v; by assumption that the leading term of v; is 
maximal.) 

e A nilpotent linear operator X: Let X be the matrix in permuted Jordan form corre- 
sponding to the rectangular n x (a” — a) Young diagram, as in Definition 6.2. (By 
construction, N = n(a” — a) so the matrix can act on a slice.) Define a bijection 
between C% and the slice under consideration by labeling the basis vectors of 
C% with r“e, in the bottom-left corner of the Young diagram for X and then in 
decreasing order up the leftmost column, then up the second-to-left column, etc. 
Since X is written in permuted Jordan form with respect to this basis, we have 
x= peg Ej\,i+n- 

With these conventions, the action of X coincides with the action of ¢ in the 
following sense: 


— tu; = u; if and only if Xu; =u; 
— tu; is outside the window if and only if Xu; = 0. 


We can now define the matrix of the permutation 7 € Sy. For each integer i with 
1 <i < K the ith column of 7 is one in the entry corresponding to the pivot of u;. 
Now append the last N — K basis vectors as columns with no descents. Though there 
are no descents in the last N — K entries, 7 is not a Grassmannian permutation: each 
of the first K entries is a descent by definition of the u;. 

For eachi € {1,2,..., K} define the Hessenberg function h as follows: 


e Iftu; =u; then h(i) = j and 
e if tu; is outside the slice then h(i) = K. 


For eachi € {K + 1,..., N} define h(i) =i. 

In the first K columns, the conditions used to define 4 are the same as those 
used to construct X and in the last N — K columns, the conditions are moot. So 
TBE H(X,h). 

We now prove that the Grassmannian Hessenberg Schubert cell defined by these 
parameters is the same as the projection of the affine Schubert cell to the specified 
slice. Corollary 5.1 says that the affine Schubert cell conditions are equivalent to 
choosing the entries of v;, v2, ..., U, freely subject to the constraint that the leading 
entry of each vector is in the position specified by 7, and generating the rest of 
the K vectors by appropriate powers t'v;. By contrast, the first K columns of the 
flags in C,,4,, are the same freely chosen v1, v2,..., UV, and powers X i v; ordered 
consistent with u1,u2,...,ux, while the last N — K columns of the flag are the 
basis vectors indicated by the last N — K entries of 7. (As before, such a flag is in 
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Hy, by construction of X.) No other flag that satisfies the Hessenberg conditions also 
satisfies the affine Schubert cell conditions because for eachi < K adding a nonzero 
linear combination of the first i — | columns of the flag to the ith column would 
change the pivot in the ith column, thus moving the flag outside of the cell C;. 

It follows that 7x (C,,7,) is the projection of the affine Schubert cell &2,, to the 
specified slice, as desired. 

This Hessenberg Schubert cell is “natural” in the sense that it doesn’t require any 
normalization to see that it is the projection of the corresponding affine Schubert 
cell. To prove part (b), we modify the cell so that the permutation flag corresponds to 
a Grassmannian permutation. (The vectors in part (b) are the Ui. U5, U3, ..., though 
this fact is not explicitly needed in the proof.) 

Define 7 to be the Grassmannian permutation obtained from the permutation 7 
by reordering the first K columns so there are no descents. 

We now confirm both that TB ¢ H(X, hjq) and that 7x (Cz,7,,) are the projection 
T(Qy) of the affine Schubert cell Q, to the slice under consideration. 

First note that TB € H(X, hjq) because by construction, the basis vectors within 
each Jordan block of X appear in the flag 7B in increasing order (though not neces- 
sarily successively). Thus the image of each of the first i columns of 7 under X is 
either zero or one of the first i — 1 columns of T. 

Now we show that 7x (Cz,n,,) is the projection of (2,, to the slice under consid- 
eration. Let c;,, cj,, ..., ¢;, be columns satisfying the following: 


e they are within the first K columns of the matrix of the flag 


e there is exactly one column corresponding to a multiple of each of e), €2,..., ey 
in the generators {t“e,} for the affine Grassmannian 
e the pivot for each column corresponds to the multiple of e;, e2,..., e, with the 


lowest power of ¢ within the first K columns of the matrix. (More technically: if 
the column c;, corresponds to r“e; then none of the first K columns of the matrix 
have a pivot in the position corresponding to t“~!e,.) 


For each i; we know that Xc;, is in the span of the first 7; columns of the matrix by 
definition of f;g and thus so are X7c;,, Mee, RG, .... The pivots of all nonzero 
elements of c;,, Xc;,, X ae, X cn X“c;,,... are distinct by construction of X. The 
pivots for different i; are distinct because they correspond to generators t“e, for 
different e,. 

This means that 7x(C+,4;,,) contains the K-dimensional space spanned by 
Ci;, Ciy, +--+, Cj, together with all images under successive powers of X of those vec- 
tors. In other words, 7x (Cz,n,,) is the K-dimensional space 7(2,) as desired. 


Proof of Theorem 7.1. Let Gr(K, N), Hig, and T be as in Proposition 7.2. 
The map 7x: FI(N) > Gr(K,N)=G/P. restricts to an isomorphism 
C,, = BuP/P for uw € Sy when py is a Grassmannian permutation so C,, = 1x (Cy). 
Since 7 is a Grassmannian permutation for Gr(K, N) and 7B is contained in jg, 
it follows that 


Cr Hig = 1K (Cott2) : 
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This is equal to 7({2) by Proposition 7.2, so setting o = T completes the proof. 


Example 7.3. We describe N and K for Examples 5.2 and 5.3. Both examples show 
vectors needed for a minimum slice. For notational convenience we pick N = 6 
in Example 5.2 and N = 9 in Example 5.3. The slice corresponding to N = 6 in 
Example 5.2 contains the generator te, so K =5, corresponding to the vectors 
V1, V2, fV1, U3, (V2. (The last vector is not shown in the example.) By contrast Example 
5.3 shows all the generators t“ ey in the slice and so the column vectors shown in the 
example are precisely the desired set of vectors, meaning K = 7. 


Example 7.4. We show 7 and 7 for Example 5.3, with T on the left. The conven- 
tion that B is upper-triangular means that the standard coset representatives for 
BoB/B € G/B have pivots in the lowest nonzero entry in each column, with free 
entries above and zeros identically to the right. This reverses the order < so the top 
row of the matrix for a flag corresponds to the largest generator t“e, and the bottom 
row corresponds to the smallest generator. 


bo by 1.4 2,4 1,1 O21 1/0 0 100 O 0 0 0 100 
bs bio bz; bg by 1 0100 610 0 0 O O00 
by bo by by 1 O O00 0010 0 0 O 00 
1.4 22.4 11 21 0 0 O/;10 000 2,1 1,1 2,4 214 10 
bz bg bh 1 O O O00 000 1 0 0O O 00 
bo by 1 0 O O O00 000 0 1 0 O 100 
a1a21 0 0 0 0 0/01 000 0 O agi a,1]0 1 
bh 1 0 0 0 O O00 000 0 0 1 O 100 
1 0 0 0 0 O O00 000 0 0 0 1 00 
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A Survey of the Shi Arrangement ®) 


Check for 
updates 


Susanna Fishel 


Abstract In [58], Shi proved Lusztig’s conjecture that the number of two-sided 
cells for the affine Weyl group of type A,_; is the number of partitions of n. As a 
byproduct, he introduced the Shi arrangement of hyperplanes and found a few of 
its remarkable properties. The Shi arrangement has since become a central object in 
algebraic combinatorics. This article is intended to be a fairly gentle introduction to 
the Shi arrangement, intended for readers with a modest background in combina- 
torics, algebra, and Euclidean geometry. 


In his 1986 paper “The Kazhdan-Lusztig cells in certain affine Weyl groups” [58], 
Jian-Yi Shi proved Lusztig’s conjecture that the number of two-sided cells for the 
affine Weyl group of type A,_, is the number of partitions of n. As a byproduct, 
he introduced the Shi arrangement of hyperplanes and found a few of its remark- 
able properties. The Shi arrangement has since become a central object in algebraic 
combinatorics. This article is intended to be a gentle introduction to the Shi arrange- 
ment, intended for readers with a modest background in combinatorics, algebra, and 
Euclidean geometry. After a review of background material in Sect. 1, we discuss 
how the Shi arrangement arose in Sect. 2, some of its marvelous enumerative prop- 
erties in Sect. 3, and some of its surprising connections to algebra in Sect. 4. For 
some brief comments on recent extensions, see Sect. 5 and for an incomplete list of 
topics we left out, see Sect. 6. 


1 Background 


In this section, we will give very brief introductions to some of the ingredients needed 
to define the Shi arrangement. 
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1.1 Root Systems and Coxeter Group Notation 


Let V be a finite dimensional real vector space with fixed inner product (|). We'll 
use A to denote a root system: a finite set of vectors in V which satisfies 


(1) AN Ra = {a, —a} for all a € A and 
(2) s,A = A forallaec A, 


where 5, is the reflection about the hyperplane with normal a. We use A* to denote 
a choice of positive roots of A, so that A = At U —A‘*, and [1 to denote the simple 
roots, which are a basis for the R-span of A. The reflections S = {sa}aen generate 
a finite reflection group W. The rank of the system and of W is the dimension of the 
space spanned by A. 

Coxeter groups generalize finite reflection groups. Let W be a group with a set of 
generators S C W. Let ms; be the order of the element st, with s,t € S. If there is 
no relation between s and t, we set ms; = oo. If W has a presentation such that 


(1) Mss = 1 
(2) fors,te S,s At, 1 < ms < ~, 


then W isa Coxeter group. We refer to (W, S) asa Coxeter system. Ifms, € {2, 3,4, 6} 
when s ¥ f¢, then the Coxeter group is called crystallographic and, if finite, is a Weyl 
group. It is also a reflection group. The product in any order of all the elements in 
S is called a Coxeter element; all Coxeter elements for a given W are conjugate and 
their order is the Coxeter number of W. 

The expression for the reflection s,, a € A, is 


(v | a) 


SD OE i a) 


for v € V. For any k € R, we can define an affine reflection Sax by 


jee i= 
(a | a) 

We define the affine Weyl group to be the group generated by all affine reflections 
Sq,¢ for a € A and k € Z. It is also a Coxeter group. Its simple reflections are the 
simple reflections S of the finite Weyl group, plus an extra reflection, so, about a 
translate of a certain other hyperplane in the arrangement. See Fig. 1. Its root system, 
for us, is the root system for the corresponding finite Weyl group. Our proofs will not 
be detailed enough to need the full set of affine roots, and we will not define them. 
Given a root system A, we write W, for the corresponding finite group. Please see 
[41] or [43] for more information. 

We put a partial order on any root system. The root poset of A is the set of 
positive roots A‘, partially ordered by setting a < 3 if 6 — ais anonnegative linear 
combination of simple roots. If the root system is irreducible, then there is a unique 
highest root relative to this ordering. We will denote this root by 6. See Fig. 2 for a 
picture of the root poset for type A4. 
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Fig. 1 The roots and Hig iy 
reflecting hyperplanes of 
affine type Ay. The reflection 
S| (resp. $2) flips the plane 
over the hyperplane H,, 9 
(Ho>,0). The reflection so 0 
reflects over Ho | : ‘ 


Oy 


A Q,4,0 


a2 


Ho Hos 


Let W be a Coxeter group. Every w € W has an expression as a product of 
elements of S: w = sj, --- s;,. If k is minimal among all expressions for w, then k is 
the length €(w). Any expression for w of length €(w) is a reduced expression. 

We will often refer to the type of a group or root system, particularly “type A” 
and “affine type A,” which are the symmetric group or affine symmetric group if we 
are referring to groups. Please see [41] for more information on the classification of 
finite reflection groups and Coxeter groups. Humphreys and [17] are good sources 
for the definitions of irreducible, Bruhat order, and other material omitted here. 


1.1.1 Type A 


We will be seeing type A often, so we’ll be a little more concrete. For A,_1, the vector 
space V is {(ay,...,dn) € R” | a, +--+ + a, = 0}. Let fe), ..., €,} be the standard 
basis of R” and (| ) be the bilinear form for which this is an orthonormal basis. The set 


Fig. 2 On the left is the nonnesting set partition 7 = {{1, 3}, {2, 4, 5}}. Next to it is the root poset 
of type A4 with the filter corresponding to 7 circled. The third figure is the partition inside the 
staircase which corresponds to 7. On the right is the corresponding Dyck path 
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of roots is A = {e; — e; |i A j} andaroota € A is positive, written a > 0,ifa € 
At = {e; — €j |i < j}. Set a; = €; — €j41; the simple roots are {a1,..., @,—1}. 
The set IT is a basis of V. 

The Coxeter group that Shi studied was the affine symmetric group om and we 
review that here. There are several possible descriptions, here we give one due to 
Lusztig [48]. It is the set of permutations w of Z such that 


(1) wé+n) = w(i)+n foralli € Z 
(2) Di w@ = ("Z') 


It’s a Coxeter group: for any i, 0 <i <n, s; corresponds to the permutation 


t ift modm Aiandt modn ¢ (i+ 1)modn 
tre yt—1 iff modn=(i+1)modn 
t+1 iftmodn=i 


The set of reflections S is {s],..., 5,1, So} and 


m~ 


Sy, S (S1, sees Sn-1s So). 


The affine symmetric group contains the symmetric group G,, as a subgroup. G,, 
is the subgroup generated by the s;, 0 <i <n. We identify G, as permutations of 
{1,...,} by identifying s; with the simple transposition (i, i + 1). We act on the 
right, as did Shi. 


1.2. A Taste of Coxeter Combinatorics, Type A 


The number of parking functions and the Catalan numbers appear in every discussion 
of the Shi arrangement. We’ ll define the parking functions when we first see them, 
in Sect. 3.5, but we’ll collect some facts on the Catalan objects here, mostly type A. 
There are an awful lot of Catalan objects, but only a few of them will appear in this 
survey. 

A partition is a finite sequence \ = (Ai, ..., ,) of positive integers in decreasing 
order: Ay > Az > ++: => A; > O. We identify a partition with its Young diagram, the 
left-justified array of boxes where the ith row from the top has 4; boxes. A box is 
called removable (respectively addable) if we can remove (respectively add) it and 
still have a diagram of a partition. We use |A| = A; + --- +A, and €(\) =r. 


1.2.1 Set Partitions 


We denote the set {1, 2,...,} by [x]. The nonempty subsets B,,..., By of [n] are 
a set partition of [n] if they are pairwise disjoint and their union is [n]. We denote 
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the set partition {B,,..., By} by B,|---|By. For example, 13|256/4 is a partition of 
[6]. The arc diagram of a set partition 7 is defined as follows: place the numbers 
1,2,..., in order on a line and draw an arc between each pair i < j such that 


e i and j are in the same block of 7, and 
e there is no k such thati < k < j andi,k, and j are in the same block. 


See Fig. 2. 

The partition 7 has k blocks if and only if it has n — k arcs. This is easy to see if 
the partition has no arcs. Consider a partition with k blocks and n — k arcs, where i 
and j are in different blocks. Suppose we add an arc from i to 7. We have joined i’s 
and j’s blocks, and we now have k — | blocks and n — k + 1 arcs. 

A set partition is called noncrossing if there does not exist i < j <k </ such 
that there is an arc from i to k and an arc from j to /. There are C,, noncrossing set 


partitions, where 
1 ("") 
C= 
n+1\n 


is the Catalan number (type A). It is called nonnesting if there does not exist i < 
j <k <Jsuch that there is an arc from i to/ and an arc from j tok. 


1.2.2. Dyck Paths 


A Dyck path of length n is a lattice path which starts at (0,0), takes only north or east 
steps of length 1, never goes below the line y = x, and ends at (n,n). A north step 
followed by an east step is called a valley of the path. 


1.2.3 Root Poset 


An ideal of a poset P is a subset I of the elements of P such that ifx € J andy < x 
then y € J. A filter is like an ideal, except that the condition becomes that x € I 
and x < y implies y € J. A subset X of the elements of P is an antichain if no two 
elements in X are comparable. An ideal is determined by its maximal elements, which 
form an antichain, just as a filter is by its minimal elements. These antichains are 
also called nonnesting partitions; in type A the bijection to the nonnesting partition 
described above is simply sending the root €; — ¢€; in an antichain to the arc from i to 
j. There are Catalan number of ideals, filters, and antichains in the root poset defined 
in Sect. 1.1. We can map a filter F in the root poset for type A,_1 to a partition 
whose diagram fits inside the staircase partition (n — 1,n — 2,..., 1) by the rule that 
€; — €; € F if and only if the box in rowi and column n + | — j is in the diagram 
of A. The minimal elements of F' correspond to the removable boxes of X. 

There are well-known bijections among all these objects; please see Stanley’s 
book on the subject [71]. 
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1.3 Deformation of Coxeter Arrangements 


A (real) hyperplane arrangement H is a set of hyperplanes, possibly affine hyper- 
planes, in a real vector space. For us, the vector space will be V, the span of some 
root system A, with a fixed inner product (|) which is W, invariant. We’ll be looking 
at connected components of a hyperplane arrangement’s complement V \ Unen H.- 
We will refer to these as the regions of the arrangement. The closure R of the region 
R is a convex polyhedron. A face of H is a nonempty set of the form R M x, where x 
is an intersection of hyperplanes in 71. The dimension of a face is the dimension of its 
affine span. See Stanley [67] for more details. A wall H of R is ahyperplane H ¢ H 
such that dim(H M R) = dim(#). The word “bounded” applied to a region has its 
usual meaning: a region is bounded if there is a real number M such that all points 
in the region are within distance M of the origin. Let t(.A) and b(A) be the number 
of regions and number of bounded regions, respectively, of the arrangement A. 

Let A be a root system. The roots (plus the integers) define a system of affine 
hyperplanes 

Hox = {ve V| la) =h). 


Note A_,,.-~ = Hox. In type A, we will sometimes write x; — x; = k instead of 


Fo 4--+aj-1,k- 
The Coxeter arrangement, also called the braid arrangement, is defined 


Coxn = {Hao : a€ At}. 


We give the regions of this arrangement the special name chambers. Each cham- 
ber corresponds to an element of W = W(A). The dominant chamber of V is 
(V2) Ho,.ot, where H,,;+ is the half-space {v € V | (v | a) > k}. the dominant 
chamber corresponds to the identity of W. It is also referred to as the fundamental 
chamber in the literature. 

The affine Coxeter arrangement is all integer translates of the hyperplanes in Cox ; 
that is, it is the whole system of hyperplanes {Ho.x}aca+xez- In this arrangement, 
each region is called an alcove and the fundamental alcove is Ao, the interior of 
Ho (‘eee H,.9', where 0 is the highestroot. A dominant alcove is one contained 
in the dominant chamber. 

We also have the m-Catalan arrangement: 


Cat’, = {Hay : a€ A,O<r <m}. 


Let W be an affine Weyl group and V the vector space spanned by its roots. W 
acts on V via affine linear transformations, and acts freely and transitively on the 
set of alcoves. In affine type A,_1, s; reflects over Hy,.9 for 1 <i <n— 1 and so 
reflects over Hy.;, where @ is the highest root. We identify each alcove A with the 
unique w € W such that A = Agw. For example, if w is the element of affine A» 
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Cath = {Hay : ae AO<r<m}. 


WAT AV AS AVA 
UT RAV AVES 
WAVAVAY VAVAV/ 
L\I\I\I\LVLVZ\ 


VAAL 
/\/\IN\IN/N/N/\ 


Hay,0 


Aa,,0 


Hoo 


Fig. 3. The Coxeter arrangement for A2 is on the left, with shaded fundamental chamber. On the 
right is all its translates, with fundamental alcove shaded 


whose reduce decomposition is sos;, then Agw is the image of Ao after reflecting 
first across Hg and then across H,,.0. See Fig. 6. 
We can be even more specific for type A,_,. The action on V is given by 


Sj(Q,,-.-,4j, G41, --+5 Ay) = (@),..-, Gj41, 4j,---, An) for i #0, and 


So(@1,---,4n) = (Qn +1, @2,...,Qn-1, 4 — 1). 


Note G6, preserves (| ), but 6, does not. 

An alcove A can be described by the hyperplanes it is between. For example, 
in Fig. 6, the alcove labeled by 20 is between H,,; and H,,.2, between H,,.9 and 
H,,,; and between Hg; and Hg. Given a positive root a, there is a unique integer 
k = k,(A) such that k < (a | x) <k+ 1 for all x € A. Let K(A) = {ka(A)}aea+ 
denote the set of coordinates for A, indexed by the positive roots. Returning to Fig. 6, 
the alcove labeled by 20 has coordinates ky, = kg = l andky, = 0. 

Shi characterized the sets of integers which can arise as K (A) for some alcove 
A; for type A in [58, Chapter 6] and for general affine Weyl groups in [59]. The 
situation in general is rather messy, but if we assume our root system is an irreducible 
crystallographic one, then Shi found (see also [11]) that a collection of integers 
indexed by the positive roots A* corresponds to an alcove if and only if 


ka +ke < kote Ska tke +1 (1.1) 


for all a, @ € At such that a + 3 € A*. We call the set K (A) the coordinates of A 
(Fig. 3). 
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2 Origin 
2.1 Kazhdan-Lusztig Cells 


We provide a bare-bones introduction to Kazhdan—Lusztig theory. For more informa- 
tion, see [17, 44, 58]. If you are willing to believe that Kazhdan and Lusztig defined 
an equivalence relation on the elements of a Coxeter group, then skip this section. We 
include this collection of definitions for completeness. We’ ll need the Hecke algebra 
§) and the Kazhdan—Lusztig polynomials in order to define the W-graph and then 
the cells. We will prove none of our claims. 

Let W be a Coxeter group and let S be the corresponding set of simple reflections. 
We first follow [44], who follow [19], for the definition of the Hecke algebra. Let A be 
the ring of Laurent polynomials in the indeterminate g!/? with integral coefficients. 
The Hecke algebra 5 = 9(W, S) is a free module over A with basis 7,,, one for each 
w € W. The multiplication is defined by the rules 


(1) Ty Ty = Tour if €(ww') = &(w) + (w’) 
2) G+ )0;—-4q) =Oifs € S; 


here £(w) is the length of w. a 
Now for the polynomials. The involution on A a +> a defined by q!/2 = q7'/? 
extends to an involution of the ring 9: 


y tule = > eal 


Kazhdan’s and Lusztig’s theorem, Theorem 2.1 in this survey, asserts the existence 
of elements C,, € 9, one for each w € W, and simultaneously defines the Kazhdan— 
Lusztig polynomials P,,,,, where y, x ¢ W. The order is the Bruhat order on W. 


Theorem 2.1 ([44]). For any w € W, there is a unique element C,, € § such that 


(—1)2O)+ 4) g2w)/2-£) Pig ly 


ysw 


where P,, € A is a polynomial in qg of degree at most $(€(w) — £(y) — 1) for 
y < w,and Py» = 1. 


Kazhdan and Lusztig used the polynomials to define a graph and from there 
the cells. Now we follow the exposition given in [17], simplified just a bit because 
we will not prove anything. For u, w € W, define jz(u, w) to be the coefficient of 
q2@)-C)-) in P, (gq) ifu < wand 5(€(w) — €(y) — 1) is an integer; otherwise, 
L(u, w) is 0. Let £(w) be the set of left descents of w: L(w) = {s € S|sw < w}. The 


directed, labeled graph ie s) 18 the graph with vertices x € W and edges x ued 


y € E. There are two types of edges in E: 
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(1) x,yeW,x Fy, either u(x, y) £0 or u(y, x) 4 0, and s € L(x) \ L(y). Let 
i be either p(x, y) or u(y, x), whichever is not 0. 
(2) Loops at x: labeled by s € S and 


_ fi ifs ¢ £0), 
OS at: Hive Lx). 


The graph ye s) has an analogous definition, using right descents of w: R(w) = 
{s € S|ws < w}. The graph Te is the superposition of the iy ,, and oe We 
describe the cells in graph theoretic terms. A directed graph is strongly connected if 
there is a directed path between all pairs of vertices. A strongly connected component 
of a directed graph is a maximal strongly connected subgraph. Finally, the /eft cells are 
the strongly connected components of Wee s)> the right cells the strongly connected 
components of Vey. sy and the two-sided cells the strongly connected components 
of TAF os 

The list of areas in math where cells appear is mind-boggling. Please see the short 
survey by Gunnells [32], for example, for references, as well as for insight into the 
geometry of the cells. The book by Bjérner and Brenti [17] explains much of the 
combinatorial connection. 


2.2 Shi Regions and Kahzdan-Lusztig Cells 


Shi was studying cells in [58]. He concentrated on the affine Weyl groups of type 
A, because of the following conjectures of Lusztig. In [48], Lusztig defined a map 7 
from G,, to partitions of n. He conjectured that for any partition of n, a !(A), a set 
of affine permutations, is in fact a two-sided cell. Lusztig also conjectured a formula 
for the number of left (or right) cells which make up the two-sided cell a~!(\). The 
description of the map is simple enough and we define here. Let w € SG, and define 
d, = d,(w) to be the maximum size of a subset of Z whose elements are noncongruent 
to each other modulo n and which is a disjoint union of k subsets each of which has its 
natural order reversed by w. The partition is given by (d,, dz — dj, ..., dy — dn-1). 


Example 2.2. Let n = 3. The permutation so is 


eet HP A OTIS ATGC oi: 
jin SPF NT OAS SS 218) * 


The set {3, 4} has its order reversed by sg and there is no larger set, so d; = 2. The 
sets {3, 4} and {2} show that d. = 3. Therefore o(sg) = (2, 1). In our notation, sos2 
iS 
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Fig. 4 The cells for affine 
A2. The yellow region is the 
two-sided cell 

o—!((1, 1, 1)), which is also 
a single left-cell. The six 


A/\/\(\o 
AVM &X 

pinkeeinsaeiteatss 7 NZ ZANT \/\. 
att AAve WV AVAVAS 
ga 
See (58, Page 98 WAVAVAY WVAVAVAY 
VAVAY A VAVAY) 
VAY AWA VAY) 
\@ /B/ 


eel BIAFG ex: 
seet\ad® 1134967 4.J* 


and o(sos2) = (2, 1) also. The identity maps to (1,1, 1) under o and s,525,, for 
example, maps to (3). 


Shi proved both of Lusztig’s conjectures, and more. Shi used the identification of 
6, with alcoves to describe the cells of affine type A. He showed that the two-sided 
cells correspond to connected sets of alcoves, one set of alcoves for each partition 
A of n. A two-sided cell is a disjoint union of left-cells. Inside the two-sided cell 
corresponding to the partition A, there is one left-cell for each tabloid of shape A. 
See Fig. 4. 

What came to be known as the Shi arrangement was not initially defined in terms 
of hyperplanes. Shi began by defining rank n sign types as triangular arrays X = 
(xij)1<i<j<n With entries from {+, —, O}. The admissible sign types correspond to 
the regions of his arrangement. He defined them as the sign types which satisfy the 
following condition: for all 1 <i <t < j <n, the triple 

Xij 
Xit Xt j 


is a member of the set G4 of admissible sign types of rank 3 (Az). G4 is the set 


rt @ Opty be ig te OR ee ox 
Sel, ae Oe ed -»>-+°-OQ°O-} 


(2.1) 
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Two comments on G4. If we order the symbols {C, +, —} as — < 0 < +, then 
Ga can be seen as the rank 3 sign types where either x12 < x13 < X23 Or X23 < x13 < 
X12, together with x3 = +, x12 = X23 = 0. The set Gy has cardinality 16, which is 
(n+1)""! forn = 3. 

Shi connected the admissible sign types to geometry using (1.1) and the map ¢. 
If K is the set of coordinates for an alcove A, then define the sign type X = ¢(A) by 


+ if kjj >0 
Xiji= O if kj; =0 
= if kjj <0. 


He then calculated the hyperplanes so that the regions defined by them were made 
up of alcoves with the same image under the map ¢. We use admissible sign type, 
region in the Shi arrangement, and Shi region interchangeably. 

Shi showed in [58] that the left-cells for affine type A are themselves disjoint 
unions of admissible sign types. Admissible sign types were not used directly in the 
proofs of the Lusztig conjectures in Shi’s monograph, but describe the structure of 
the cells. They have taken on a life of their own. 

Later, in [60], Shi extended the definition of admissible sign types, thereby gen- 
eralizing the Shi arrangement. This is the definition we give below. 

We give the definition for any irreducible, crystallographic root system A. When 
the root system is type A,_;, we will sometimes write Shi, instead of Shi,. 


Definition 2.3. The Shi arrangement Shi, is the collection of hyperplanes 


{Hon | ae AT,O<k < 1}. 


Example 2.4. All alcoves in the region labeled by 


O 
+ -— 


in Fig. 5 have positive coordinate kj. = ky,, have negative coordinate k,,, and have 
the coordinate kg equal to zero. Likewise, all three coordinates of all alcoves in the 
region labeled by 


+ 
++ =F 
are positive. See Sect. 1.3 for the definition of coordinates of an alcove. 


We mention here that in the case where the Coxeter graph of the system contains 
an edge with a label greater than 3, it is not true that all the left-cells of the affine 
Weyl group are unions of admissible sign types. It may be conjectured that it holds 
for any affine Weyl group of simply-laced type. The cells in affine D, have been 
explicitly described by Shi in [63], so the conjecture may not be difficult to verify. It 
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Fig. 5 The Shi arrangement for type Az. See [58, Page 102]. Each region has been labeled with its 
sign type. See Example 2.4 


is known that any left-cell in the lowest or highest two-sided cell of any irreducible 
affine Weyl group forms a single admissible sign type; see Shi [61, 62]. We thank 
Jian- Yi Shi for this information. 

By a dominant region of the Shi arrangement, we mean a connected component 
of the hyperplane arrangement complement V \ Uy. sni, 7 that is contained in 
the dominant chamber. Both the formula for the number of regions in the whole 
arrangement and for the number of dominant regions are intriguing and will be 
discussed in the next section. 


3 Enumeration 


The Shi regions have been counted multiple times. We discuss four different 
approaches to enumerating them. 
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3.1 The Number of Shi Regions, Part 1 


Shi concentrated on the admissible sign types in Chapter 7 of his book [58], where 
he introduced the arrangement for type A. 

He enumerates them for type A by considering the alcove closest to the origin 
in each region. We’ll call this the minimal alcove of the region and denote it Ar 
if the region is R. Shi called such an alcove the shortest alcove [58, Section 7.3]. 
He characterized Ag using left descents. Left descents are key to the definition of 
cells, so it is not surprising that they appear in the description of minimal alcoves. 
Basically, an alcove is minimal if any reflection which brings it closer to the origin 
flips it out of the region. 


Example 3.1. See Fig. 6. Let w = s,525,59. This w has two left descents, s; and 
Sj, and both s;w = 525,59 and syw = 515289 are in different regions the minimal 
permutation w. On the other hand, w = 5; 5259 is not minimal and indeed €(s;w) < 
£(w) and s;w = 52So is in the same region as w. 


Every alcove corresponds to an affine permutation. We’ll call the affine permu- 
tations whose alcoves are minimal minimal permutations. Shi showed that the col- 
lection of alcoves corresponding to the inverses of minimal permutations is exactly 
a scaled version of the fundamental alcove. See Fig. 6. Thus to calculate the number 
of regions in his newfound arrangement, he calculated the number of alcoves in this 
scaled fundamental alcove. He calculated something a bit more general: if the fun- 
damental alcove is expanded by the positive integer m, then it is made up of m#™) 
alcoves. The alcoves corresponding to the inverses of minimal permutation land in 
the fundamental alcove scaled by n + 1, which showed that there are (n + 1! 
regions in the Shi arrangement of type A. The expression (n + 1)""! pops up fre- 
quently in combinatorics and algebra; see [35] for their connection to qg, t-Catalan 
numbers, for example. 


Fig. 6 On the left, the minimal alcove of each Shi region is shaded. The dominant ones are shaded 
yellow, the rest gray. We’ve labeled the alcoves by the corresponding affine permutation, where we 
use i for s; for space reasons. The alcoves corresponding to the inverses are drawn on the right 
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Shi’s enumeration of the regions is perhaps more complicated than the others 
described here. However, his discovery that the inverses of the minimal permutations 
correspond to a simplex is worth the price of admission. The minimal alcoves have 
been useful in other enumeration; see [12, 28], for example. For another example, 
Hohlweg, Nadeau, and Williams, in [39], generalize the Shi arrangement to any 
Coxeter group (and beyond!) and conjecture that the inverses of the analogues of 
minimal permutations form a convex body. See also Sommers [66] where the simplex 
was generalized to what is now called the Sommers region. Thomas and Williams 
[75] show that the set of alcoves in this region, and by extension the Shi regions, 
exhibit the cyclic sieving phenomenon. 

In [60], Shi generalized sign types to other affine Wey] groups. He defined sets 
analogous to (2.1) for other types. The hyperplane arrangements were still given by 
Definition 2.3, but now he considered root systems other than type A. He used the 
map ¢ on alcoves and described the sign types which arose and again characterized 
the element in each region with the minimal number of hyperplanes separating it 
from the origin. As above, we identify elements w © W with Aow and refer to w as 
minimal if its alcove is minimal. The fact that 


U Agu! 


w minimal 


is a simplex is not just a type A phenomenon. Shi proved it for other affine Weyl 
groups and used it to prove that there are ( + 1)” regions, where h is the Coxeter 
number of the system. 

Shi counted the number of regions in the dominant chamber for affine Weyl 
groups in [64]. He calls the admissible sign types corresponding to regions in the 
dominant chamber -sign types. For types A, B, C, and D, he finds a bijection from 
®-sign types to filters in the root poset for At. Here is a technical detail: Shi finds 
the bijection to the positive coroots (AT)Y, which we won’t define, then mentions 
that it has the same type as At except when At has type B and C. He deals with 
types B and C separately. We will continue using At. He further maps the filters 
to subdiagrams of certain Young diagrams; see Sect. 1.2.3. For example, for type 
A, the subdiagrams are those of partitions whose diagrams fit inside the staircase 
shape. In the exceptional cases, he enumerates increasing subsets directly. He shows 
the @-sign types for affice Weyl groups are enumerated by the Catalan numbers, 
although Shi does not mention them. 

A few more words are in order on this important bijection to filters in the root 
poset. The key proposition, from Section 1.2 of [64], follows (using roots instead of 
coroots). 


Proposition 3.2. Assume that X = (Xq)aca+ is a At-tuple with X. € {+, O}. 
Then X is an @-sign type if and only if the following condition on X holds: if 
a, 3 € At satisfy 8 > a and X, =+4, then Xz = +. 


We'll use type A as an example. In the set Gy in (2.1), there are five triples 
which contain only © and +, but there are eight which are possible. The condition 
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in Proposition 3.2 rules the other three out, proving sufficiency. For necessity, Shi 
uses induction to reduce to the rank two case and shows that the five @-sign types 
in G satisfy the condition. 


3.2 Interlude 


We’ ll need these standard definitions for Sects. 3.3 and 3.4. See [70] for the definitions 
of the rank function p and Mobius function j: of a poset. 


Definition 3.3. [70] Let P be a finite graded poset with 0. Let p be its rank function 
and n the rank of P. Define the characteristic polynomial p(x) of P by 


xp(x) = > uO, tx". 


teP 


Definition 3.4. [70] Let A be a hyperplane arrangement in a vector space V and 
let L(A) be the set of all nonempty intersections of hyperplanes in A. Include V 
itself, by considering it as the intersection over the empty set. Order L(A) by reverse 
inclusion, so that Ois V. 


See Fig. 7 for an example of the poset of intersections. 

If the intersection of all the hyperplanes in A is nonempty, then L(A) is a lattice. 
The intersection of all hyperplanes in Shi, is empty and L(Shi,) will only be a 
meet semi-lattice. It is finite and graded by p(t) = n — dim(t), where n = dim(V). 
The characteristic polynomial of an arrangement is 


wua= > noo 


teL(A) 


+2 +2 aril srl +2 aril 


21 =t2= 23 21 =2@2 =23+1 2 =a,2+1=23 2) -l=2.=23 @)—-1l=22=2341 21 =22-1=23 


Fig. 7 The poset of intersections for type Az. For example, the node labeled x; = x2 = x3 + 1 
represents Hy, 0 M Has,0 MN Ha3,1. See [37, Page 36]. The numbers in the circle above or below each 


element is the Mébius function of the interval from the element to V = 0. The Poincaré polynomial 
is Pa,, (x) = (1+ 3x)? 
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and the Poincaré polynomial is 


Pax) = D> 20,0 (-x). 


teL(A) 


The Poincaré polynomial is a rescaled version of the characteristic polynomial. 

The characteristic polynomial is invaluable for studying hyperplane arrangements, 
thanks to a theorem of Zaslavsky [76, 77]. See also Stanley’s notes on hyperplanes 
[69]. In Sect. 1.3, we defined r(A) and b(A) be the number of regions and number 
of bounded regions of the arrangement A. 


Theorem 3.5 ([77]). Let Abe an arrangement in an n-dimensional real vector space. 
Then 


(A) = (-1)"x.4(-1) = Pa(1) 
b(A) = (-1)™* 4x4) = Pa(-1). 


3.3. The Number of Shi Regions, Part 2 


Headley [37, 38] calculated the Poincaré polynomial Psp;, of the Shi arrangement 
for an irreducible root system. He found a recursion for its coefficients, which we 
will now present. 

Let Shia be the subarrangement of Shi, consisting of all hyperplanes which 
contain the origin. For Y € L(Shi,), let Wy be the group generated by the reflection 
through all the hyperplanes containing Y. For a polynomial p(t), let [t*]p(t) be the 
coefficient of r* in p(t). 


Lemma 3.6 ([38]). For Y € L(Shia), let Wy, x --- Wy.m be the decomposition 
of Wy into irreducible Coxeter groups. Let S; = Shiw,, be the Shi arrangement 
associated to the Coxeter group Wy,;. Then 


[t] Psnig(t) = U0] 7 Ps,(t) +++ Ps, (t). 


YeEL(Shi,):rank(Y)=k 


Headley uses induction on the number of generators to determine every coefficient 
except the leading one. For this, he relies on Shi’s enumeration and the relationship 
between the Poincaré polynomial and the number of regions. His analysis is done 
case by case for each Coxeter type. His theorem is 


Theorem 3.7. Let A be an irreducible crystallographic root system, with Coxeter 
number / and rank n. Then 


Psni,(t) = A +ht)". 
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In type A, the argument for calculating the coefficients is simple enough to repeat: 
he matches an element Y of the intersection poset L(Shi,) with the set partition 
B= (B,,..., By) of [n+ 1] by 


Y =NM{x; — x; =0: i, j are in the same block of B}. 


In this case, Wy is isomorphic to Ajp,j-1 X --- x Az 
Therefore, by induction and Lemma 3.6, 


j-1 andrank(Y) =n+1—~m. 


m 


[t*] Psnig(t) = » Be Beg 
Partitions of [n + 1] inton + 1 —k blocks 
(3.1) 
In his thesis [37], he used Lagrange inversion to calculate 


Pon =n + " ) 
n—k 


Later, in [38], he recognized the sum in (3.1) to be the number of labeled forests on 
n+ 1 vertices of n + 1 — k trees and used [51]. In both his thesis and later paper, 
he showed that the coefficient of t* in Psp; a(t) and (1 + (n+ 1))” are the same for 
1 <k <n-—1.Thensince the degree of Psp; , (t) ism and since Psy; ,(1) = (n + 2)” 
by Shi’s result, he showed 


Psni,(t) = A+ (n+ It)" = + ht)" (3.2) 


in A,. 


3.4 The Number of Shi Regions, Part 3 


Crapo and Rota [23, Chapter 16] described the critical problem: let S be a set of 
points in an n-dimensional vector space V,, over the field F, with q elements. The set 
S must not contain the origin. Find the minimum number c of projective hyperplanes 
H,..., H- with the property that the intersection H,1...H.9S is null. They 
were able to solve the problem using the poset of intersections and characteristic 
polynomial. 

Athanasiadis [8] turned Crapo and Rota’s theorem around and used it to calcu- 
late the characteristic polynomial of subspace arrangements. Blass and Sagan [18] 
had previously used a similar idea, but not for all subspaces and not for the Shi 
arrangement. We present first the the Crapo and Rota theorem, then describe how 
Athanasiadis used it to get his hands on the characteristic polynomial for the Shi 
arrangements for irreducible crystallographic root systems. 
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Theorem 3.8 ({23]). The number of linearly ordered sequences (Lj, L2,..., Lx) 
of k linear functionals in V,, which distinguish the set S is given by p(q*) where p(v) 
is the characteristic polynomial of the geometric lattice spanned by the set S. 


Athanasiadis needed to count the number of n-tuples (x1,...,%n) € te which 
satisfy x; Ax; and x; Ax; +1 for i < j. The argument is simple (and lovely) 
enough in type A,_; for the full Shi arrangement that we reproduce it here. See also 
[70]. 

We first solve a related problem. Find the number of ways there are to place n 
labeled balls in g unlabeled boxes, where 


(1) the boxes are in a circle, 

(2) there is never more than one ball in a box, and 

(3) if i < j, then ball i is not placed in the box immediately following, in the 
clockwise direction, the box holding ball /. 


There will be g —n empty boxes, so first place them in a circle. There is one 
way to do that. There are now g — n spaces between the empty boxes, where the 
boxes holding the balls will go. By cyclic symmetry, there is one way to place the 
box holding the 1-ball. Then there are (¢ — n)"—! ways to place the rest of the boxes 
holding balls in the empty spaces. It is enough to pick the space between empty boxes: 
to avoid violating condition (3), the boxes between a consecutive pair of empty boxes 
must placed in increasing order of the labels on the balls inside. That is, our final 
answer to the related problem is (¢ — n)""!. 

Now back to counting n-tuples. We are essentially done, if we think of each n- 
tuple representing a distribution of n labeled balls into a circle of g labeled boxes, 
where the distribution satisfies conditions (2) and (3). We place the ball labeled i in 
box x;. We need only label the boxes, and there are g ways to do this. Thus there are 
q(q —n)"~' n-tuples which satisfy x; A x; and x; ¢ x; +1 fori < j and we have 
that the characteristic polynomial is yzsni,,) = 4q(q —n)""!. 

Crapo and Rota’s finite field method has since been used to calculate other char- 
acteristic polynomials. See Armstrong [5], Armstrong and Rhoades [7], and Ardila 
[3], for example. See Athanasiadis [13] for reciprocity results for the characteristic 
polynomial for the Shi arrangement. 


3.5 The Number of Shi Regions, Part 4 


Pak and Stanley, in [67], give a bijection from Shi regions (type A) to parking 
functions, which refines (3.2). It is proved to be a bijection in [69, Lecture 6]. A 
parking function of length n is a tuple of nonnegative integers (p1,..., Pn) such 
that when rearranged in nondecreasing order and relabeled as b) < by <--- < Dy, 
then b; < i — 1. Parking functions generalize inversion vectors of permutations. Pak 
and Stanley recursively defined the label \(R) of a region R. We use the description 
given in [69, Lecture 6]. 
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Let Ro be the fundamental alcove Ag. Set A(Ro) = (0, --- , 0). Suppose we have 
labeled the region R and its label A(R) is (a), ..., Gn). 


e Ifthe regions R and R’ are separated by the single hyperplane H with the equation 
x; —x; =0,i < j, and if R and Ro lie on the same side of H, then \(R’) = 
(a1,..-, 4-1, + 1, dj41-.., Gj 10j,Qj41,+-+,n)- 

e Ifthe regions R and R’ are separated by the single hyperplane H with the equation 
x; —x; =1,i < j, and if R and Ro lie on the same side of H, then A(R’) = 
(a1,...4j 1, Gj, Giz. ---,@j-1, 4j + 1, aja, wey yp). 


The bijection generalizes the well-known bijection from permutations to inversion 
vectors [70, Chapter 1]. Although the map \ is simply stated, the proof that it is a 
bijection is not simple. To show that the labeling is a bijection, Stanley encodes each 
region as a permutation and antichain pair. He builds the inverse map step-by-step 
from the parking functions to the pairs. The summary by Armstrong [5, Theorem 3] 
of the proof that the Pak—Stanley map is bijective is particularly good. Recall that 
the filters/antichains/ideals in the root poset for type A correspond to partitions in a 
staircase, and define the non-inversions of a permutation w to be the pairs (i, 7) such 
thati < j and w(i) < w(j). Then the proof can be summarized as showing that the 
Shi regions are in bijection with pairs (w, Z) where w € G,, and Z is an ideal in the 
root poset A* such that the minimal elements of Z, which are labels in the valleys 
of the Dyck path corresponding to Z, are non-inversions of w (Fig. 8). 


Fig. 8 The Shi regions labeled by parking functions, using the Pak—Stanley labeling. A label is 
nondecreasing if and only if the region is in the dominant region 
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The Pak and Stanley bijection from regions to parking functions (m = 1) case 
can be composed with a bijection from trees to parking functions. The number of 
regions R for which i hyperplanes separate R from the region Ro is equal to the 
number of trees on the vertices 0,..., with () — i inversions. The pair (i, /), 
where | <i < j, is an inversion for T if the vertex j lies on the unique path in T 
from 0 to i. See [67, Theorem 5.1]. 

We mention a few papers which build on the Pak—Stanley bijection. Duertes and 
Guedes de Oliveira, for example, further analyze this bijection in [25]. Rincén [56] 
extends the Pak—Stanley labeling to the poset of faces of the Shi arrangement. See 
also Sect. 5. 


3.6 More 


Believe it or not, there are still other wonderful proofs concerning the number of 
regions. 

For example, Athanasiadis and Linusson [14] defined a bijection, different from 
Pak and Stanley’s, from parking functions to the Shi regions (type A). Theirs gives 
a simple proof of the number of regions. Their bijection was generalized to type C 
by Mészaros in [50]. In [7, Section 5.2] there is another proof of the formula for 
the number of regions, using Armstrong’s and Rhoades’ ceiling diagrams, which we 
define in Sect. 3.7. The ceiling diagrams are related to the diagrams Athanasiadis 
and Linusson used. Armstrong, Reiner, and Rhoades [6] define nonnesting parking 
functions using the root poset and permutations from the finite Weyl group and label 
the Shi regions with these. Their definition can also be used for types which are not 
crystallographic. Other, more recent and more general bijections include [16, 40] for 
example. 


3.7. The Ishand the Shi 


We'll start off by writing the g, t-Catalan polynomial combinatorially: 


Cc. (q, t) = > quam ponent (3.3) 


7 


where the sum is over all Dyck paths of length n. The (q, t)-Catalan polynomials 
are remarkable generating functions coming from representation theory. They have 
been intensely studied since their introduction by Garsia and Haiman in [30]. See 
Haglund’s monograph [35, Chapter 3] for more information. There are the same 
number of dominant Shi regions as there are Dyck paths, and one of Armstrong’s 
results in [5] (and the one we’ll describe) was to transfer the statistics area and bounce 
to dominant regions. His statistics are actually for all regions. The statistic shi will 
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Fig.9 The Ish arrangement forn = 3 is on the left. The Shi arrangementis on the right, where region 


R is labeled by the pair (shi(R), ish(R)). The dominant regions are also labeled by g (3)—shi(R) pish(R) | 
The sum of the monomials is C3(q, t) = ge t qt + qt + qt? + £3 


correspond to the statistic area and shi(R) is defined as the number of hyperplanes 
which must be crossed on a trip to the region R from Ro = Ao. The statistic ish 
is defined using a second hyperplane arrangement, the Ish arrangement Zsh,,. It is 
defined for type A and is a deformation of the Coxeter arrangement. Let A be the set of 
roots for type A, so a; = €; — €;41 aS in Sect. 1.1.1. Denote a; + aj4) +--+ + Qn-1 
by a;. Then the definition of the Ish arrangement is 


Tshyn = Coxn U {Ha,«|1 <j<n-1,ke({l,2,...,n—j}} 
= Cox, U{xj —x, =klk € {1,...,n-j}, l<j<n-]}. 


The ish statistic is defined on Shi regions using the hyperplanes in Zsh,. Let R be 
region of the arrangement Shi,, with minimal alcove Ar. There is a unique w € G,, 
such that Ar = Agw. The affine permutation w has a unique factorization w! wy 
[17], where w,; € G, and w! is a minimal length coset representative, which we 
won’t define. What is important for us is that Agw/ is an alcove in the dominant 
chamber since w/ is a minimal length coset representative. Then ish(R) = ish(Ar) 
is the number of hyperplanes in Zsh, which must be crossed in traveling from Ag to 
Au! (Fig. 9). 

Each dominant Shi region R corresponds to a Dyck path 7r. Armstrong showed 
that () — shi(R) = area(7p) and ish(R) = bounce(7R) Notice that the ish and shi 
statistics are defined on all regions, not just the dominant ones. Armstrong was able 
to show they agree with bounce and area on all diagonally labeled Dyck paths. See 
[35, Chapter 5] and [5, Section 3]. 

Armstrong and Rhoades concentrated on properties of the Ish arrangement, espe- 
cially its uncanny similarities to the Shi arrangement, in [7]. Their definition of the 
arrangement changes just a bit: replace x; — x, =i by x1 —XxX,-j4; =n —it+l. 
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Their main theorem is for deleted versions (more general) of the arrangements (see 
Sect. 6), but we'll stick with the full arrangements. That is, G is the complete graph 
in this survey. We need to define a few terms before we can state the main theorem. 
The wall H of a region R is called a ceiling if it does not contain the origin and if 
the origin and R are not separated by H (they lie in the same half-space of H). The 
regions of both Zsh, and Shi, are convex, so every region has a recession cone: 


R(R)={vEeV:ivtRC R}. 


The cone is closed under nonnegative linear combinations and has a dimension. The 
dimension of 7(R) is called the degrees of freedom of R. It’s worth mentioning that 
the region R is bounded if and only if R(R) = {0}. 

A simplified version of their main theorem can now be stated: 


Theorem 3.9 ([7]). Let c and d be nonnegative integers. The Zsh, and Shi, have 
the same 


(1) characteristic polynomial, 
(2) number of dominant regions with c ceilings, and 
(3) number of regions with c ceilings and d degrees of freedom. 


We are not presenting their theorem in its full generality, and as written here, (1) 
was proved in [5]. 

We want to define the ceiling diagrams for the Shi arrangement because they show 
the properties of the corresponding region so clearly. We will also need variations 
on the root poset, which they use to prove Theorem 3.9. First the definition of the 
Shi ceiling diagram of a region R. Suppose the region is in the chamber Dw, where 
D is the dominant chamber and w € G,,. Then we define the set partition or: there 
is an arc from i to j, i < j, in the diagram of op if and only if the hyperplane 
Xw(i) — Xwij) = 1 is a ceiling of R. We draw the Shi ceiling diagram (w, or) by 
placing the arc diagram for or above w(1), w(2),..., w(m). See Example 3.10. 
Armstrong and Rhoades show that og is a nonnesting set partition. The number of 
arcs is c. What about d? Let d’ be the number of k, 1 < k <n — 1 where there is no 
arc covering the space between k and k + 1; that is, the number of k where for which 
there is noi < j such that i < k < j and there is an arc from i to j. For example, 
d' = 0 for the set partition in Fig. 2 and d’ = 1 for the set partition 12|35|4. Then set 
d =d' + 1. Additionally, the recession cone 7(R) can be read from the diagram. 

There is still a key point: for a fixed w € G,, both the regions and the ceiling 
diagrams are in bijection with antichains in At(w). The poset A*(w) is the first 
variation on the root poset: 


At (w) = {Xwi) — Xwj) = 1: wl) < w(j)}. 


The elements of A*(w) are the affine hyperplanes in the Shi arrangement which 
intersect Dw. The partial order on the hyperplanes is given by 


Xwii') — Xw(j) = 1S Xwey — Xwyy = 1 
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Fig. 10 The Shi arrangement for Az. The chamber (D)w is labeled by w 


£1 —%3 =1 v1 —-@3 =1 


ie = ay = Il C3 9) =) = Il 


Fig. 11 On the left, the poset A*(e). The poset A*(s2) is on the right 


if wi) < w(i’) < w(j’) < w(J). The partial order is defined so that ceilings of any 
Shi region R, R € Dw, are the maximal elements of an order ideal. The number c 
shows up as the number of these maximal elements. 
We’ll need the second variation on At, Y*(w), for discussing the Ish arrange- 
ment: 
Wt(w) = (41 —xj =i: w'@) <w'({)}. 


Its elements are the Ish hyperplanes that intersect w D and its partial order is chosen 
so that the ceilings of any Ish region R, R € wD, are minimal elements of a filter, 
and c for the region is the number of these minimal elements. 


Example 3.10. This example refers to Figs. 10 and 11. In Fig. 10, the chamber Dw 
is labeled by w € G3. We’ve picked two Shi regions to consider in this example-the 
ones we have labeled R; and Ro. 
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R, is in the chamber Dw for w = e, the identity. There are three hyperplanes 
of the form x; — x; = 1 which intersect Dw and the poset A*(w) is on the left in 
Fig. 11. The hyperplane x2 — x3 = | is aceiling for R, and the region is also labeled 
with its ceiling diagram in Fig. 10. 

The region R) is in the chamber Dw for w = s2 and poset A*(s2) is on the right in 
Fig. 11. The poset At (s2) has three ideals, corresponding to the three Shi regions in 
Ds. Our region Ro has ceiling x} — x3 = Xw1) — Xw(2) = 1 and we have placed the 
arc diagram for the partition 12|3 above w(1)w(2)w(3) to build the ceiling diagram. 


The relationship between the filters and the regions is bijective in the Ish case, 
just as between ideals and regions in the Shi case. The posets A*(w) and Y*(w) 
are dual to each other when w is the identity permutation e. The final step in the 
proof of Theorem 3.9, part (3), is simply to send an order ideal in A*(e) to the 
corresponding filter in W*(e). Since the maximal elements in the ideal become the 
minimal elements in the filter, c is preserved. There are also ceiling diagrams for the 
Ish arrangement, but we won’t define them. 

To prove Theorem 3.9, part (3), Armstrong and Rhoades used ceiling partitions, 
which are set partitions of [n]. We now define a simplified version of them. First 
suppose R is a Shi region. The ceiling partition 7p has an arc from i to j,i < j, 
if and only if the hyperplane x; — x; = 1 is a ceiling of R. Next suppose we have 
an Ish region R. Its ceiling partition has an arc from i to j,i < j, if and only if 
x; — x; =i is aceiling of R. The definition of the ceiling partition does not depend 
on the chamber of R for either arrangement. Surprisingly, the distribution of the set 
partitions is the same for the Ish and Shi arrangements. 


Theorem 3.11 ([7]). Let A be either the Ish or the Shi arrangement. Let 7 be a 
partition of [n] with k blocks and let 1 <d <k. 


(1) The number of regions of A with ceiling partition 7 is 


n! 


(n—k+1)! 
(2) The number of regions of A with ceiling partition 7 and d degrees of freedom is 


d(n—d—1)\(k—1)! 
(n—k—I1\(k—d)! 


To obtain the number of regions with c ceilings and d degrees of freedom, thereby 
proving Theorem 3.9, part (3), sum the expression in Theorem 3.11, part (2), over 
all partitions 7 with k = n — c blocks. 

For space reasons, we cannot include the arguments here for Theorems 3.9 
and 3.11. This is a shame, because we thereby don’t present evidence for their 
observation [7]: 


The Ish arrangement is something of a “toy model” for the Shi arrangement (and other 
Catalan objects). That is, for any property P that Shi, and Zshy share, the proof that Zshy 
satisfies P is easier than the proof that Shi,, satisfies P. 
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Many of the theorems of [7] are proved bijectively by Leven, Rhoades, and Wilson 
in [47]. 


3.8 Extended Shi Arrangement 


In [54], Postnikov and Stanley introduced the extended Shi arrangement of type 
An-1: 
Shit = {Hox | ae A*,-m+1<k <m}. 


This kind of extension is sometimes denoted by Fuss, as in Fuss-Catalan [4]. Up 
until now, we have been discussing m = 1. Postnikov and Stanley show that Shi?” 
has (mn + 1)""! regions. They fix m, set f, = t(Shi’") to be the number of regions, 
and show that the exponential generating function 


x” 
f= >. hn 


n>=0 


satisfies 
fae an 


The extended Shi arrangement is a special case (a = m,b = m + 1) of what they 
named truncated affine arrangements; see [54, Section 9] for more details. The 
dominant regions of the m-Shi are the same as the dominant regions of the m-Catalan. 

Many of the other enumerative treats of the Shi arrangement generalize well to the 
extended Shi arrangement. For example, Stanley [68] labeled the m-Shi regions with 
m-parking functions of length n using an extended version of the bijection described 
here in Sect. 3.5. Stanley defined m-parking functions of length n. He replaced the 
condition that “b; <i — 1” in the definition of parking function (see Sect. 3.5) by 
“b; < m(i — 1).” If we set d(R) = area(R) (see Sect. 3.7), then Stanley’s bijection 
showed [68, Corollary 2.2] that 


yg = 3 ghitetPa (3.4) 
R 


where the sum on the left is over all regions in Shi)” and the sum on the right is over 
all m-parking functions of length n (Fig. 12). 

In 2004, Athanasiadis wrote two papers on the extended Catalan arrangement for 
crystallographic A, concentrating on the dominant regions. The Catalan arrange- 
ment has more hyperplanes than the Shi arrangement, but it has the same dominant 
regions, so we record his results here in terms of the m-Shi arrangement. We’ll need 
a definition. The Narayana numbers (type A) are given by [53] 
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Fig. 12. The Shi¥ 
arrangement. There are 
49 = (2-3+ 1)? regions 


N..= 1 n\(n—1 
me ee TNE & OS 


They refine the Catalan numbers by counting the Dyck paths of length n with k 
peaks. In other words, C, = ae Nn“. Athanasiadis 


(1) generalized and extended the Narayana numbers, finding what they enumerate 
in terms of dominant m-Shi regions; 

(2) counted the number of m-Shi regions in the dominant chamber, generalizing 
Shi’s result described in Sect. 3.1; and 

(3) used co-filtered chains of ideals in the root poset to describe the dominant m-Shi 
regions. 


We will describe (3) in a bit more depth. Let AT = Jy D J) Dh D--- DI, beaa 
decreasing chain Z of ideals in AT, set J; = I, for alli > m, and set J; = At \ Jj. 
The chain Z is a co-filtered chain of ideals of length m if 


(1) CF + Tj) nN At = Tis j and 
) GIOIA Cds 


is true for all indices i, 7 > 1 withi + j < m. The coordinates of the chain are 
k.(Z) = max{ki tha +---+k-r:a=(,+---+, with 6; € J, for all i}. 
Athanasiadis showed that 
ka (Z) + kg(Z) S ka+p(Z) < ka(Z) + kp) +1 (3.5) 
whenever a, 3, a + 3 € A*. Equation (3.5) generalizes Shi’s bijection between fil- 
ters in the root poset and dominant Shi regions (see Sect. 3.1). Finally, to define the 


fundamental m-Shi region associated to Z, he sets Rz to be the set of points x € V 
which satisfy 
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(1) (a|x)>k,ifae i and 
(2) 0 < (a| x) <k,ifa € J, forO < k < m. The coordinates of the ideal are then 
the coordinates of a region. 


Certain elements in an ideal are called indecomposable. These elements correspond 
to the walls of Rz which separate Rz from Ro, and take the place of peaks in Dyck 
paths when defining the Narayana numbers in terms of Shi regions. 

Here we mention a few other enumerative results concerning the regions of the 
extended Shi arrangement. Any fixed hyperplane in the m-Shi arrangement is dis- 
sected into regions by the other hyperplanes in the arrangements. Fishel, Tzanaki, 
and Vazirani enumerate the number of regions for certain fixed hyperplanes in type 
A in [27]. Fishel, Kallipoliti, and Tzanaki [26] defined a bijection between dominant 
regions of the m-Shi arrangement in type A, and dissections of an m(n + 1) 4+ 2- 
gon. These dissections represent facets of the m-generalized cluster complex. In 2008, 
Sivasubramanian [65] gave combinatorial interpretations for the coeffiencients of a 
two-variable version of Stanley’s distance enumerator (3.4) in type A for m = 1. 
Forge and Zaslavsky study the integral points in [7m] that do not lie in any hyper- 
plane of the arrangement [29]. Thiel resolves a conjecture of Armstrong [4, Conjec- 
ture 5.1.24] on the distribution of floors and ceilings in the dominant regions of the 
m-Shi arrangements for all types. See Sect. 3.7 for the definition of ceiling. 


4 Connections 


4.1 Decompositions Numbers and the Shi Arrangement 


The dominant regions make an appearance in the study of decomposition numbers 
for certain Hecke algebras. To describe this appearance, we’ll first need a host of 
combinatorial definitions, then we’ll indicate briefly how these arose from algebra, 
and finally we’ll relate this back to the Shi arrangement. We thank Matthew Fayers 
for not only pointing out this connection, but carefully explaining it. 


4.1.1 Combinatorics 


Here we define n-cores, review some well-known facts about them, and review the 
abacus construction, which will be useful for us. Details can be found in [42]. 

The (k, /)-hook of an integer partition \ consists of the box in row k and column 
I of X, all the boxes to the right of it in row k together with all the nodes below it 
and in column /. The hook length Aten of this box is the number of boxes in the 
(k, 1)-hook. Let n be a positive integer. An n-core is a partition \ such that n { hi. ) 
for all boxes \. An n-regular partition has no (nonzero) n parts which equal each 
other. For example, (7, 6, 6, 6) is not 3-regular. We’ll sometimes use p or e instead 
of n, depending on the context. The definition is the same (Fig. 13). 
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Fig. 13. The Young diagram 
of the partition 9 5 3 2 1 
A = (5, 2, 1, 1, 1). The 

hooklengths are the entries in 
the boxes of its Young 

diagram. The partition \ is a 3 
4-core but not a 5-core 


ol 
= 


Fig. 14 On the left, the 
beads are placed on the 


positions labeled by the first : : 
column hook-lengths of =) dD ©) =) 
d= (5, 2, 1, 1, 1). On the * Re 

right is an equivalent abacus, (a 


where all bead positions have 
been shifted by C = —1 


The 3-numbers of the partition A are the hook lengths from its first column. The 
(-numbers can be displayed on an abacus: a p-abacus is a diagram with p runners, 
labeled 0, 1,..., p — 1. Runner i has positions labeled by integers pj +i, for all 
j € Z. We make a p-abacus for 4 by placing a bead at position (;, for each 3-number 
G, of X and at all negative positions. We say two p-abaci are equivalent if we can 
change one to the other by moving the bead at position 7 to position i + C for some 
C € Zand for all positions i where there is a bead. The positive integer p is arbitrary 
for now, but will be related to the characteristic of a field when we see abaci in their 
algebraic context. See the 4-abacus in Fig. 14. 

We can give an equivalent description of p-core partitions: a partition \ is an 
p-core if and only if whenever there is a bead at position j of its p-abacus, there is 
also a bead at position j — p [42]. 

Suppose we have a partition which is not an p-core. Then there is at least one 
bead at a position j of its p-abacus which can be pushed up into the vacant position 
j — p. This gives us the p-abacus of another partition. We can repeat this until no 
beads can be pushed up, at which point we have the p-abacus of an p-core. The final 
partition y is called the p-core of \ and the number of beads we moved in called 
the p-weight of X. With a little work, which we won’t do, it is possible to show that 
[Al = ll + wp. 

The last combinatorial ingredient we need are residues for the boxes of a partition. 
Let n be a positive integer. We call the box inrowi, column j ak-boxif (j — i) mod n 
is equal to k. 
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Fig. 15 The beads are 

placed on a 4-abacus on the 

positions labeled by the first : 

column hook-lengths of aT) 

A = (5,2, 1, 1, 1). The 

partition A is not a 5-core, =) 

since there are gaps on the (0) 
a 

10 


5-abacus. If we push the 
beads in positions 5 and 9 up 


to fill in the gaps, we obtain (11) (14) 
the 5-abacus of the empty : : 
partition 
Fig. 16 The Young diagram 
of the partition 0 1 2 3 0 
A = (5, 2, 1, 1, 1). The 
entries are the residues 3 0 
modulo 4 

2 

1 

0 


4.1.2 Algebra 


It is time to say a few words about how the combinatorics from the last section relate 
to algebra (Fig. 15). 

Roughly speaking, the p-regular partitions of n index the irreducible modules D* 
of F'S,,, where p is the characteristic of the field F. The p-blocks are the equivalence 
classes of a certain equivalence relation on the irreducible modules. By Nakayama’s 
celebrated conjecture, and Brauer and Robinson’s theorem, two irreducibles D* and 
D* belong to the same p-block if and only \ and yz have the same p-core. Thus 
each p-block is labeled by a p-core and p-weight w. The p-weight keeps track of 
the difference between the p-core labeling a block and the partitions in the block: 
the p-core is a partition of n — pw. The weights are nonnegative integers. We will 
assume w is at least one, because the blocks where w = 0 are singletons consisting 
of the p-core (Fig. 16). 

Scopes [57] investigated the classes of p-blocks under Morita equivalence. She 
characterized families of Morita equivalent p-blocks using the p-core and p-weight 
which label p-blocks. Suppose B is p-block for F'G,, labeled by weight w and p-core 
y. Let k be an integer at least as large as w, and suppose that in a p-abacus for y, 
there is a runner i which has k more beads in positive positions than runner (i — 1) 
has. Now move k beads from runner 7 to runner i — | in sucha p-abacus for 7. This 
new abacus determines another p-core, say y. See Fig. 17. The operation changes 
the size of the partition: |y| — k = |y|. We are glossing over details here involving 
the 3-numbers. Let B be the block of FG,_; labeled by w and 7. Scopes took the 
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Fig. 17 On the left, the 
abacus for y = (6, 3, 1, 1, 1), 


on the right, the abacus for 

7 = (5, 2, 1, 1, 1). We have @ 
moved k = 2 beads from DG 
runner 2 to runner 1 


(0) 
( 4) 
8) 


transitive closure of the relation B ~ B and showed that within an equivalence class, 
the p-blocks have the same decomposition matrix, among other results. 

Richards was studying the decomposition numbers for the Hecke algebra; see 
Sect. 2.1 for at least the definition. He used the classes from the Scopes equivalence 
on e-cores, where e depends on the characteristic p of the field and the element g used 
in the definition of the Hecke algebra. He called the classes families. Richards was 
interested in these families because the blocks of the Hecke algebras for G, and for 
G6, corresponding to y and ¥ respectively have essentially the same decomposition 
numbers [55]. 

Richards wanted to count such families. He built the following pyramid 
{u@v}o<u<v<e—1 for an e-core y based on -y’s e-abacus. Note the similarity in shape to 
admissible sign types and to the arrangement of roots in a staircase shape diagram in 
Fig. 2. Fori = 0, 1,..., e — 1, let p; be the position of the first free space on runner 
i. Arrange these e numbers in ascending order and relabel as pp < py <--- < Pe-1. 
If0 <u <v <e-—1, then p, — p, is a positive integer not divisible by e. We may 
use any e-abacus for 7; it doesn’t affect the set of differences. Richards defined the 
pyramid of numbers by 


w—1 if0< p,— py <e 

w—2 ife < py — py < 2e 

1 if (w — 2)e < py — Py < (w— Le 
0 if (w — le < py — Pv 


where w is the weight. Richards proved that two e-cores are in the same family if 
and only if they have the same pyramid and that there are exactly 


1/ ew 

e\e-1 
families. What’s more, he characterized the triangles of numbers which form a pyra- 
mid. 
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To show the connection to the Shi arrangement, we transform ,a,, into ,d@,, by 
yay + yay = w — 1. Then Richard’s Proposition 3.4 becomes 


Proposition 4.1 ((55]). Let e > 2 and w > 0, and forO <u <u <e-—JI1letO< 
,ady < w— 1. Then the ,a, form a pyramid if and only if for allO<u<t<v< 
e-l, 


ud + pay OF 4, + 4a, +1. if both of these have all entries no bigger than w — 1 


Qe 
| 


w-1 otherwise. 


Please see Example 4.2 for the calculation of a few pyramids from cores. 
Let’s examine the case e = 3 and w = 2. There are five families. We choose five 
3-cores {@, (1), (2), 1, 1), (, 1, 1)} and calculate their pyramids: 


i) 


oo ofo of: roldlh 


The pyramids are all different, so we have found all the families. If we look back at 
the set G in Sect. 2.2 and consider the subset where all entries are either + or C, we 
see a similarity to the pyramids (replace + with 1). This is true in general. Richard’s 
proposition is the type A version of (3.5) from Sect. 3.8. The pyramid is also an 
admissible sign type for a dominant m-Shi region of type A, where m = w — 1. 


4.1.3 Geometry 


We'll just say a few more words about the geometry here. Richard’s pyramids have 
connected the core partitions to regions. We’ll describe Lascoux’s [46] well-known 
bijection between n-cores and certain elements of 6, and by extension, between 
n-cores and alcoves in the dominant chamber. Please see Lapointe’s and Morse’s 
paper [45] for details. We describe the bijection, as another way of seeing why core 
partitions pop up here. An n-core partition may have several removable boxes of a 
given residue or it may have several addable boxes of a given residue, but it will 
never have both addable and removable boxes of the same residue. Given an n-core 
partition and the generators so, 51, ..., 5)—1 of G,,, let s;(A) be the partition where 
all boxes of residue i have been removed (added) if there are removable (addable) 
boxes. Any n-core partition can be expressed as w(4). See Fig. 18 and Example 4.2. 
We associate the n-core w(@) with the alcove Aow. 

We mention that Fishel and Vazirani mapped partitions which are both n and 
nm + 1 cores to dominant regions in the m-Shi arrangment of type A,_; in [28] 
using abacus diagrams and the root lattice. 


Example 4.2. First, we construct the two pyramids for the partition (4,2), one 
each for w = 2 and w = 3. From Fig. 19, we see that (p, p}, Ps) = (po. Pi, p2) = 
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Be / a aw 


a 


Fig. 18 The dominant chamber for Az with 3-cores. The thick solid lines are hyperplanes from the 
1-Shi arrangement, the dashed ones are added to create the 2-Shi arrangement 


Fig. 19 On the left, a 

3-abacus diagram for (4, 2). E 

On the right, a diagram for : : ; : 
(3, 1). The right abacus is the © é os a i 
result of moving two beads $ $ 3d $ =) 


in the left abacus, as 


described in Sect. 4.1.2 (0) (1) Y (0) 3 = 
a : Si 

( & 

8 ) 


(0, 1, 8). When w = 2, the pyramid is (944, y@,, ;@,) = (0, 1, 0) and when w = 3, 
it 1S (94, y@), ;4_) = (0, 2, 0). We also calculate the coordinates/admissible sign 
type of the region containing (4, 2)’s alcove. For m = 1 (w = 2), (9G, 9, 4.) = 
(1, 0, 1) and for m = 2 (w = 3), (oy, o@), 14>) = (2, 0, 2). Additionally, 


0 


) = sos2( 


598281 50(9) = sos2s) ( 0 Ly = sof 1/2) —|0|1]2/0 


2|0 


where the entries in the boxes are their residues mod 3. We have placed (4, 2) in the 
alcove corresponding to 59525159. See Fig. 18. 

Consider the 1-Shi region containing (4, 2)’s alcove. The minimal alcove of this 
region is labeled with the 3-core (2). To reach this region from Ag, we must cross 
only one translate each of Hg; and H,,,9, and no translates of Hy, 9. This is reflected 
in pd, = 1, ,a4, = 1, and )a, = 0 respectively. Next, we consider the 2-Shi region 
containing (4, 2)’s alcove. This region, whose minimal element alcove is labeled by 
(4, 2) itself, is separated from the fundamental alcove by two translates for each of 
Ho,; and H,,,9, and no translates of H,,.9, reflected in the m = 2 pyramid for (4, 2). 
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We repeat the calculations for the 3-core (3, 1), whose 3-abacus is on the right in 
Fig. 19. We calculate 


0) ~s5,(9 1) — : 1|2 


528189) = S251 ( 


so we have placed (3, 1) in the alcove corresponding to s25;59. The w = 2 pyramid 
for (3, 1) 1s (9a, y@, 1a) = (0, 1, 0), the same as the pyramid for (4, 2), indicating 
that its alcove will be in the same 1-Shi region as the alcove labeled by (4, 2). Its 
w = 3 pyramid is (9d, 9@,, 14.) = (0, 2, 1), which is not the same as the w = 3 
pyramid for (4, 2), and indeed, their alcoves are in different 2-Shi regions. 

Lastly, we mention that the action of moving beads as described in Sect. 4.1.2 
corresponds to flipping the alcove containing the core of the original abacus over a 
hyperplane, to the alcove containing the core obtained through the bead move. 


4.2 Finite Automata and Reduced Expressions 


Headley used the Shi arrangement to build an automaton which recognizes reduced 
expressions. A language CL is a subset of the set B* of words in a given finite alphabet 
B. For us, the alphabet will be the set S of a Coxeter group W and the language will 
be reduced expressions. 

A finite state automaton is a finite directed graph, with one vertex designated as 
the initial state Sp and a subset of vertices as final states, and with every edge labeled 
by an element of B. We call the vertices states. A word w € CL is accepted by the 
automaton if the sequence of edge labels along some directed path starting at So and 
ending at a final state is equal to w. 

Headley was not the first nor the last to construct an automaton to accept reduced 
words; see Bjorner and Brenti [17], Hohlweg, Nadeau, and Williams [39], and Gun- 
nells [33], for instance. However, Headley realized that if s;,5;, - - - s;, is reduced, then 
the fundamental alcove Ag and Aos;,5;, -- - S;, lie on the same side of the hyperplane 
fixed by s;, if and only if s;,5;,5;, ---5;, is reduced. He built his automaton on this 
observation. The key lemma is 


Lemma 4.3. [37] Let W be an irreducible affine Weyl group with root system A. 
Let R be a region of the Shi arrangement. If R and Ao lie on the same side of the 
hyperplanes fixed by s € S, then Rs lies in a single region. 


The states of his automaton are the regions of the Shi arrangement. The funda- 
mental alcove Ao, which is also a Shi region, is the initial state. All states are final. 
Let s € S, and let R be a region. If R and Apo are on the same side of the hyperplane 
fixed by s € S, then let R’ be the region containing Rs and place an arrow from R 
to R’, labeled by s. If R and Ao are not on the same side of the hyperplane, then 
R has no outgoing arrow labeled by s. Headley not only showed that the language 
accepted by this automaton is the set of all reduced words, he showed that if W is 
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Fig. 20 Headley’s automaton based on the Shi arrangement. It accepts reduced words in 63. Each 
state is labeled with the affine permutation corresponding to the minimal alcove of the region. We 
use i instead of s;. Solid arrows represent an edge labeled by so and dashed (respectively dotted) 
edges represent edges labeled s; (respectively sz). See Example 4.4 


the affine symmetric group, the automaton has the minimal number of states. There 
were actually two automata in Headley’s thesis. The first produced a nice generating 
function, but it has more states. 


Example 4.4. This example refers to Fig. 20. The path 


0 2 1 0 1 2 
e>0->02-> 21> 10 101 > 1012 
represents the expression 595251595152, Which is reduced. Since sos;sos; is not 


reduced, there is no path which starts at e and follows solid, dashed, solid, dashed 
arrows. 
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4.3 More Connections 


This connection is to the filters in At, not to the Shi arrangement directly. In [20— 
22], Cellini and Papi investigate ad-nilpotent ideals in a Borel subalgebra. They 
associate each ideal to a filter in At and also to an element of the affine Weyl 
group. Very roughly speaking, they use the Cartan decomposition L = H @ N and 
N= Dc a+ Lo and the definition of an ad-nilpotent ideal as an ideal contained in 
N to define the antichain 


A; ={aeé At: Lya CT} 


which defines a filter. See also Suter [73]. Dong extends Cellini and Papi’s work from 
Borel subalgebras to parabolic subalgebras in [24]. He uses deleted Shi arrangements, 
which we don’t address in this survey. Panyushev [52] developes combinatorial 
aspects of the theory of ad-nilpotent ideals, giving a geometric interpretationfor the 
number of generators of an ideal, for example. 

Gunnells and Sommers study Dynkin elements, which we won’t define, in [34]. 
They define N-regions, which turn out to be unions of Shi regions. A simplified 
version of their main theorem is that if x is the point of minimal Euclidean length in 
the closure of an N-region, then 2x is a Dynkin element. 


5 Further Developments 


We briefly mention a few recent results. In [31], Gorsky, Mazin, and Vazirani devel- 
oped “rational slope” versions of much of what has been discussed here. A tuple 
(b,,..., b,) of nonnegative integers is called an M/n-parking function if the Young 
diagram with row lengths equal to bj, ... , b, arranged in decreasing order fits above 
the diagonal in ann x M rectangle. We’ve stated it a bit differently than in Sects. 3.5 
and 3.8, but if we let M@ = n-+ 1 and M = mn + | respectively and reverse the order 
of the tuple, we obtain the same functions. Gorsky, Mazin, and Vazirani defined M- 
stable permutations to take the place of minimal permutations of Shi regions and 
generalized the Pak—Stanley bijection, as well as another map defined by Anderson 
[2] in her study of core partitions. They conjectured their generalization of the Pak— 
Stanley map is injective for all relatively prime M and n. In 2017, McCammond, 
Thomas, and Williams [49] proved the conjectures in [31]. Additionally, Gorsky, 
Mazin, and Vazirani connect their maps to the combinatorics of g, t-Catalan polyno- 
mials. Sulzgruber [72] built on [31] by finding the coordinates of the M-stable per- 
mutations, generalizing (3.5). Thiel [74] extended their work to other types, among 
other results. 

As mentioned in Sect. 3 Hohlweg, Nadeau, and Williams generalized the Shi 
arrangement to any Coxeter group, using n-small roots, and then to indefinite Coxeter 
systems. They also investigated automata. 
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6 Themes We Haven’t Included 


We give a short and incomplete list of topics we have not discussed. 


(1) The Shi arrangement is free. Either see original article by Athanasiadis [9] or 
his excellent summary [10]. Abe, Suyama, and Tsujie [1] show that the Ish 
arrangement is free. 

(2) In graphical arrangements or deleted arrangements, some of the hyperplanes 
have been removed. We survey only the complete Shi arrangement. 

(3) We have no discussion of the connections to the torus O /A+ mh)Q, where 
O is the coroot lattice of a root system, (mh + DO is its dilate, and h is the 
Coxeter number of the root system. See Athanasiadis [12] or Haiman [36] for 
more information. 

(4) The enumeration of bounded regions has nice results, which we have not dis- 
cussed. See Athanasiadis and Tzanaki [15], for example and Sommers [66]. 
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Variations on a Theme of Schubert ®) 
Calculus one 


Maria Gillespie 


Abstract In this tutorial, we provide an overview of many of the established com- 
binatorial and algebraic tools of Schubert calculus, the modern area of enumerative 
geometry that encapsulates a wide variety of topics involving intersections of linear 
spaces. It is intended as a guide for readers with a combinatorial bent to under- 
stand and appreciate the geometric and topological aspects of Schubert calculus, and 
conversely for geometric-minded readers to gain familiarity with the relevant com- 
binatorial tools in this area. We lead the reader through a tour of three variations on 
a theme: Grassmannians, flag varieties, and orthogonal Grassmannians. The orthog- 
onal Grassmannian, unlike the ordinary Grassmannian and the flag variety, has not 
yet been addressed very often in textbooks, so this presentation may be helpful as an 
introduction to type B Schubert calculus. This work is adapted from the author’s lec- 
ture notes for a graduate workshop during the Equivariant Combinatorics Workshop 
at the Center for Mathematics Research, Montreal, June 12-16, 2017. 


1 Introduction 


Schubert calculus was invented as a general method for solving linear intersection 
problems in Euclidean space. One very simple example of a linear intersection prob- 
lem is the following: How many lines pass through two given points in the plane? 

It is almost axiomatically true that the answer is 1, as long as the points are distinct 
(otherwise it is 00). Likewise, we can ask how many points are contained in two lines 
in the plane. The answer is also usually 1, though it can be 0 if the lines are parallel, 
or oo if the lines are equal. 

In higher dimensions, the answers may change: in three-dimensional space, there 
are most often zero points of intersection of two given lines. One can also consider 
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more complicated intersection problems involving subspaces of Euclidean space. 
For instance, how many planes in 4-space contain a given line and a given point? 
Generically, the answer will be 1, but in degenerate cases (when the point is on the 
line) may be oo. 

It seems that the answers to such problems are often 1, 0, or oo, but this is not 
always the case. Here is the classic example of Schubert calculus, where the answer 
is generically 2: 


Question 1.1. How many lines intersect four given lines in three-dimensional 
space? 


Hermann Schubert’s nineteenth-century solution to this question! would have 
invoked what he called the “Principle of Conservation of Number” as follows. Sup- 
pose the four lines /,, /2, /5, 14 were arranged so that /, and /, intersect at a point P, 
ly and J; intersect at Q and none of the other pairs of lines intersect and the planes 
p1 and p2 determined by /;, /2 and /3, l4, respectively, are not parallel. Then p; and 
p2 intersect at another line a, which necessarily intersects all four lines. The line 3 
through P and @Q also intersects all four lines, and it is not hard to see that these are 
the only two in this case. 

Schubert would have said that since there are two solutions in this configuration, 
there are two for every configuration of lines for which the number of solutions is 
finite, since the solutions can be interpreted as solutions to polynomial equations 
over the complex numbers. The answer is indeed preserved in this case, but the lack 
of rigor in this method regarding multiplicities led to some errors in computations in 
harder questions of enumerative geometry. 

The following is an example of a more complicated enumerative geometry prob- 
lem, which is less approachable with elementary methods. 


Question 1.2. How many k-dimensional subspaces of C" intersect each of k - (n — 
k) fixed subspaces of dimension n — k nontrivially? 


Hilbert’s 15th problem asked to put Schubert’s enumerative methods on a rigorous 
foundation. This led to the modern day theory known as Schubert calculus. 

The main idea, going back to Question 1.1, is to let X; be the space of all lines 
L intersecting /; foreach i = 1, ..., 4. Then the intersection X¥; M X2M X3M Xq4 is 
the set of solutions to our problem. Each X; is an example of a Schubert variety, an 
algebraic and geometric object that is essential to solving these types of intersection 
problems. 


'See [49] for Schubert’s original work, or [45] for a modern exposition on Schubert’s methods. 
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1.1 “Variations on a Theme” 


This tutorial on Schubert calculus is organized as a theme and variations.” In particu- 
lar, after briefly recalling some of the necessary geometric background on projective 
spaces in Sect. 2 (which may be skipped or skimmed over by readers already familiar 
with these basics), we begin in Sect.3 (the “Theme’) with the foundational ideas of 
Schubert calculus going back to Schubert [49]. This includes a rigorous development 
of Schubert varieties in the Grassmannian, the set of all k-dimensional subspaces of 
a fixed n-dimensional space, and a more careful geometric analysis of the elemen- 
tary intersection problems mentioned in the introduction. We also establish the basic 
properties of the Grassmannian. Much of this background material can also be found 
in expository sources such as [22, 28, 39], and much of the material in the first few 
sections is drawn from these works. 

In Variation | (Sect. 4), we present the general formulas for intersecting complex 
Schubert varieties and show how it relates to calculations in the cohomology of the 
Grassmannian as well as products of Schur functions. Variation 2 (Sect.5) repeats 
this story for the complete flag variety (in place of the Grassmannian), with the 
role of Schur functions replaced by the Schubert polynomials. Finally, Variation 3 
(Sect. 6) explores Schubert calculus in the “Lie type B” Grassmannian, known as the 
orthogonal Grassmannian. 

There are countless more known variations on the theme of classical Schubert 
calculus, including Grassmannians in the remaining Lie types, partial flag varieties, 
and Schubert varieties over the real numbers. There is also much that has yet to be 
explored. We conclude with an overview of some of these potential further directions 
of study in Sect. 7. 


2 Background on Projective Space 


The notion of projective space helps clean up many of the ambiguities in the ques- 
tion above. For instance, in the projective plane, parallel lines meet, at a “point at 
infinity’. It also is one of the simplest examples of a Schubert variety (Fig. 1). 

One way to define projective space over a field k is as the set of lines through the 
origin in one higher dimensional space as follows. 


Definition 2.1. The n-dimensional projective space P; over a field k is the set of 
equivalence classes in k"*! \ {(0, 0, ..., 0)} with respect to the relation ~ given by 
scalar multiplication, that is, 


(x0, X1, sane on) ied (yo, Yi; nA) 


7A play on words that references the shared surname with musical composer Franz Schubert, who 
also lived in Germany in the nineteenth century. 


3Photograph of the train tracks downloaded from edupic.net. 
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Fig. 1 Parallel lines meeting 
at a point at infinity 


if and only if there exists a € k \ {0} such that ax; = y; for all i. We write (xo: 
xX, 1+++1!2X,) for the equivalence class in pk containing (xo, ..., X,), and we refer to 
(Xo 1X1 2 +++ i Xp_) as a point in P* in homogeneous coordinates. 


Note that a point in P}, is a line through the origin in k"*'. In particular, a line 
through the origin consists of all scalar multiples of a given nonzero vector. 

Unless we specify otherwise, we will always use k = C and simply write P” for 
P< throughout these notes. 


Example 2.2. In the “projective plane” P?, the symbols (2: 0: 1) and (4: 0: 2) 
both refer to the same point. 


It is useful to think of projective space as having its own geometric structure, rather 
than just as a quotient of a higher dimensional space. In particular, a geometry is often 
defined as a set along with a group of transformations. A projective transformation 
isamap f : P” > P” of the form 


f (Xo 2 Xp e+ Xn) = Oot Yi t Yn) 


where for each i, 
Yi = AjnX0 + GjiX1 + +++ GinXn 


for some fixed constants a;; € C such that the (7 + 1) x (n+ 1) matrix (a;;) is 
invertible. 

Notice that projective transformations are well defined on P” because scaling all 
the x; variables by a constant c has the effect of scaling the y variables by c as well. 
This is due to the fact that the defining equations are homogeneous: every monomial 
on both sides of the equation has a fixed degree d (in this case d = 1). 
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2.1 Affine Patches and Projective Varieties 


There is another way of thinking of projective space: as ordinary Euclidean space 
with extra smaller spaces placed out at infinity. For instance, in P', any point (x : y) 
with y 4 Ocan be rescaled to the form (¢ : 1). All such points can be identified with 
the element t € C, and then there is only one more point in P', namely (1 : 0). We 
can think of (1 : 0) as a point “at infinity” that closes up the affine line C! into the 
“circle” P'. Thought of as a real surface, the complex P! is actually a sphere. 

Similarly, we can instead parameterize the points (1 : t) byt € C' and have (0 : 1) 
be the extra point. The subsets given by {(1 : t)} and {(¢ : 1)} are both called affine 
patches of P', and form a cover of P', from which we can inherit a natural topology 
on P' from the Euclidean topology on each C!. In fact, the two affine patches form 
an open cover in this topology, so P! is compact. 

As another example, the projective plane P” can be written as the disjoint union 


{x:y: D}u{@:1:0}uUf1:0:08}=Cuc'uc, 


which we can think of as a certain closure of the affine patch {(x : y : 1)}. The other 
affine patches are {(x : 1: y)} and {(1 : x : y)} in this case. 
We can naturally generalize this as follows. 


Definition 2.3. The standard affine patches of P” are the sets 
(Qo: ties i te-pilitapi-: tt )}SC" 
fori =0,...,n. 


An affine variety is usually defined as the set of solutions to a set of polynomials 
in k” for some field k. For instance, the graph of y = x? is an affine variety in R’, 
since it is the set of all points (x, y) for which f(x, y) = y — x? is zero. 


In three-dimensional space, we might consider the plane defined by the zero locus 
of f(x, y,z) =x+y-+z, that is, the set of solutions to f(x, y, z) = 0. Another 
example is the line x = y = z defined by the common zero locus of f(x, y, z) = 
x — yand g(x, y,z) =x —z. 

Recall that a polynomial is homogeneous if all of its terms have the same total 
degree. For instance, x” + 3yz is homogeneous because both terms have degree 2, 
but x7 — y + 1 is not homogeneous. 
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Definition 2.4. A projective variety is the common zero locus in P” of a finite set 
of homogeneous polynomials fi (xo, ..., Xn), -.++ fr (Xo, -.+, Xn) in P”. We call this 
variety V(f\,..., f,). In other words, 


Vifi,---> fr) = {0 +++ Gn) | fi(Qo +++! d,) = 0 for all 7}. 


Remark 2.5. Note that we need the homogeneous condition in order for projective 
varieties to be well defined. For instance, if f(x, y) = y —x? then f(2,4) =0 
and f (4,8) £0, but (2: 4) = (4: 8) in P'. So the value of a nonhomogeneous 
polynomial on a point in projective space is not, in general, well defined. 


The intersection of a projective variety with the 7th affine patch is the affine variety 
formed by setting x; = | in all of the defining equations. For instance, the projective 
variety in P” defined by f(x: y:z) = yz — x? restricts to the affine variety defined 
by f(x, y) = y — x? in the affine patch z = 1. 

We can also reverse this process. The homogenization of a polynomial f (xo, ..., 
Xn—1) inn variables using another variable x, is the unique homogeneous polynomial 
g(Xo +++! Xp—-1 : Xn) With deg(g) = deg(f) for which 


g(xXo port Xn-1: 1) = f (Xo, wee Xn-1)- 


For instance, the homogenization of y — x? is yz — x”. If we homogenize the equa- 
tions of an affine variety, we get a projective variety which we call its projective 
closure. 


Example 2.6. The projective closure of the parabola defined by y — x? — 1 = Ois 
the projective variety in P* defined by the equation yz — x? — z* = 0. If we intersect 
this with the y = 1 affine patch, we obtain the affine variety z — x7 — z? = 0 in the 
x, z variables. This is the circle x? + (z — 5)? — i and so parabolas and circles are 
essentially the same object in projective space, cut different ways into affine patches. 

As explained in more detail in Problem 2.3 below, there is only one type of 


(nondegenerate) conic in projective space. 


Remark 2.7. The above example implies that if we draw a parabola on a large, flat 
plane and stand at its apex, looking out to the horizon we will see the two branches of 
the parabola meeting at a point on the horizon, closing up the curve into an ellipse.* 


2.2 Points, Lines, and m-Planes in Projective Space 


Just as the points of P” are the images of lines in C”*', a line in projective space can 
be defined as the image of a plane in k”*!, and so on. We can define these in terms 
of homogeneous coordinates as follows. 


4Unfortunately, we could not find any photographs of parabolic train tracks. 
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Definition 2.8. An (1 — 1)-plane or hyperplane in P” is the set of solutions (xo : 
-++1X,) to a homogeneous linear equation 


AoXo + 1X1 ++ ++ + ayXn = O. 


A k-plane is an intersection of n — k hyperplanes, say ajoxXo + Gj1X1 +-+* + GinXn = 
0 fori = 1,...,n —k, such that the matrix of coefficients (q;;) is full rank. 


Example 2.9. In the projective plane P", the line /; given by 2x +3y+z=0 
restricts to the line 2x + 3y + 1 = 0 in the affine patch z = 1. Notice that the line /, 
given by 2x + 3y + 2z = O restricts to 2x + 3y + 2 = 0 in this affine patch, and is 
parallel to the restriction of /; in this patch. However, the projective closures of these 
affine lines intersect at the point (3 : —2 : 0), on the z = 0 line at infinity. 

In fact, any two distinct lines meet in a point in the projective plane. In general, 
intersection problems are much easier in projective space. See Problem 2.3 below to 
apply this to our problems in Schubert calculus. 


2.3 Problems 


2.1. Transformations of P': Show that a projective transformation on P! is uniquely 
determined by where it sends 0 = (0: 1), 1 = (1: 1), andoo = (1: 0). 

2.2. Choice of n + 2 points stabilizes P”: Construct a set S of n + 2 distinct points 
in P” for which any projective transformation is uniquely determined by where 
it sends each point of S. What are necessary and sufficient conditions for a set 
of n + 2 distinct points in P” to have this property? 

2.3. All conics in P’ are the same: Show that, for any quadratic homogeneous poly- 
nomial f(x, y, z) there is a projective transformation that sends it to one of x7, 
x? + y*, or x? + y? +2”. Conclude that any two “nondegenerate” conics are 
the same up to a projective transformation. 


(Hint: Any quadratic form can be written as xAx’ where x = (x, y, z) is the 
row vector of variables and x? is its transpose, and A is a symmetric matrix, with 
A = A’. It can be shown that a symmetric matrix A can be diagonalized, i.e., 
written as BD BT" for some diagonal matrix D. Use the matrix B as a projective 
transformation to write the quadratic form as a sum of squares.) 

2.4 Schubert Calculus in Projective Space: The question of how many points are 
contained in two distinct lines in C” can be “projectivized” as follows: if we 
ask instead how many points are contained in two distinct lines in P?, then the 
answer is always | since parallel lines now intersect, a much nicer answer! 
Write out projective versions of Questions 1.1 and 1.2. What do they translate to 
in terms of intersections of subspaces of one higher dimensional affine space? 
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3 Theme: The Grassmannian 


Not only does taking the projective closure of our problems in P” make things easier, 
it is also useful to think of the intersection problems as involving subspaces of oo 
rather than k-planes in P”. The definition of the Grassmannian below is analogous 
to our first definition of projective space. 


Definition 3.1. The Grassmannian Gr(n, k) is the set of all k-dimensional sub- 
spaces of C". 


As in projective spaces, we call the elements of Gr(n, k) the “points” of Gr(n, k), 
even though they are defined as entire subspaces of C”. We will see soon that Gr(n, k) 
has the structure of a projective variety, making this notation useful. 

Every point of the Grassmannian can be described as the span of some k indepen- 
dent row vectors of length n, which we can arrange in ak x n matrix. For instance, 
the matrix 

0 -1 -3 -l 6 -4 55 
0 1 3 2 -7 6 -5 
0 0 0 2 =2. A =2 


represents a point in Gr(7, 3). Notice that we can perform elementary row operations 
on the matrix without changing the point of the Grassmannian it represents. We will 
use the convention that the pivots will be in order from left to right and bottom to 
top. 


Exercise 3.2. Show that the matrix above has reduced row echelon form: 
00 0 0 0 0 1 
0 0 0 1 *« « OF, 
Oo 1 « O x x O 


where the * entries are certain complex numbers. 
We can summarize our findings as follows. 


Fact 3.3. Each point of Gr(n, k) is the row span of a unique full-rank k x n matrix 
in reduced row echelon form. 


The subset of the Grassmannian whose points have a particular reduced row 
echelon form constitutes a Schubert cell. Notice that Gr(n, k) is a disjoint union of 
Schubert cells. 


3.1 Projective Variety Structure 


The Grassmannian can be viewed as a projective variety by embedding Gr(n, k) in 
P@-! via the Pliicker embedding. To do so, choose an ordering on the k-element 
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subsets S of {1, 2, ..., } and use this ordering to label the homogeneous coordinates 
Xs of pG@)-!, Now, given a point in the Grassmannian represented by a matrix M, 
let xs be the determinant of the k x k submatrix determined by the columns in the 
subset S. This determines a point in projective space since row operations can only 
change the determinants up to a constant factor, and the coordinates cannot all be 
zero since the matrix has rank k. 

For example, in Gr(4, 2), the matrix 


0 O 1 2 
1 -3 0 | 

has Pliicker coordinates given by the determinants of all the 2 x 2 submatrices formed 
by choosing two of the columns above. We write x;; for the determinant formed 
0 2 
—3 3 
dinates (x12 : X13 : X14 : X23 ! X24 : X34) then the image of the above point under the 
Pliicker embedding is (0: —1: —2:3:6: 3). 

One can show that the image is a projective variety in pG)-!, cut out by homo- 


geneous quadratic relations in the variables xs known as the Pliicker relations. See 
[17], pg. 408 for details. 


columns i and j, so for instance, x24 = det ( ) = 6. If we order the coor- 


3.2. Schubert Cells and Schubert Varieties 


To enumerate the Schubert cells in the Grassmannian, we assign to the matrices of 
the form 


000 0 0 0 1 
0 0 0 1 « « +O 
Oo 1 «x O « «x O 


a partition, that is, a nonincreasing sequence of nonnegative integers \ = (\1,..., 
Ax), as follows. Cut out the k x k staircase from the upper left corner of the matrix, 
and let A; be the distance from the edge of the staircase to the | in row i. In the 
example shown, we get the partition \ = (4, 2, 1). Notice that we always have \; > 
Ag = = Ag. 


eee 0 0 0 1 
0 0|0 1|/* * O 
mij* O* * 0 


Definition 3.4. The size of a partition A, denoted |A|, is }°; 4;, and its length, 
denoted /(A), is the number of nonzero parts. The entries \; are called its parts. 
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Remark 3.5. With this notation, Schubert cells in Gr(n, k) are in bijection with the 
partitions \ for which /(A) < k and A; <n—k. 


Definition 3.6. The Young diagram of a partition is the left-aligned partial grid 
of boxes in which the ith row from the top has \; boxes. 


For example, the Young diagram of the partition (4,2, 1) that came up in the 
previous example is shown as the shaded boxes in the diagram below. By identifying 
the partition with its Young diagram, we can alternatively define A as the complement 
inak x (n —k) rectangle of the diagram jy defined by the right-aligned shift of the 
* entries in the matrix: 


Since the k x (n — k) rectangle is the bounding shape of our allowable partitions, 
we will call it the ambient rectangle. 


Definition 3.7. For a partition \ contained in the ambient rectangle, the Schubert 
cell Q25 is the set of points of Gr(7, k) whose row echelon matrix has corresponding 
partition \. Explicitly, 


QS ={V € Gr(n, k) | dim(V N (e1,..., e,)) =iforn—k+i-Aj <r<n-—k+i-— Hj44}- 
Here e,_;+; is the ith standard unit vector (0,0,...,0,1,0,...,0) with the | in 
the ith position, so e; = (0, 0,..., 1), e2 = (0, 0,..., 1, 0), and so on. The notation 
(e1,..., e,-) denotes the span of the vectors e),..., é;. 


Remark 3.8. The numbers n — k + i — 4; are the positions of the 1’s in the matrix 
counted from the right. 


Since each * can be any complex number, we have Q5 = C"-)-IA as a set, and 
so 
dim(Q25) = k(n — k) — |Al. 


In particular, the dimension of the Grassmannian is k(n — k). 
We are now in a position to define Schubert varieties as closed subvarieties of 
the Grassmannian. 


Definition 3.9. The standard Schubert variety corresponding to a partition A, 
denoted Q2), is the set 


Q, = {V € Grin, k) | dim(V 0 (e1, .... €n-kti-,)) = B- 
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Remark 3.10. In the topology on the Grassmannian, as inherited from projective 
space via the Pliicker embedding, the Schubert variety Q) is the closure (2° of the 
corresponding Schubert cell. We will explore more of the topology of the Grassman- 
nian in Sect. 4. 


Note that we have dim(Q,) = dim(QS) = k(n — k) — |A| as well. 


Example 3.11. Consider the Schubert variety Q-> in P° = Gr(6, 1). The ambi- 
ent rectangle is a | x 5 row of squares. There is one condition defining the points 
V € Qe, namely dim(V | (e1, e2, e3, e4)) = 1, where V is a one-dimensional sub- 
space of C°. This means that V is contained in (€1,..., 4), and so, expressed in 
homogeneous coordinates, its first two entries (in positions es and eg) are 0. 

Thus, each point of 2-74 can be written in one of the following forms: 


(0:0: 1:*:%: >) 
(0:0:0:1:*: x) 
(0:0:0:0: 1: x) 
(0:0:0:0:0: 1) 


It follows that Q-74 can be written as a disjoint union of Schubert cells as follows: 


Qo = 2245 22 LU Qe LU Qe 


In fact, every Schubert variety is a disjoint union of Schubert cells. See the problems 
at the end of this section for details. 


We may generalize this construction to other bases than the standard basis 
€1,..--, ,, OF More rigorously, using any complete flag. A complete flag is a chain 
of subspaces 

Fe:0=Fyop CF C::-Ck,=C" 
where each F; has dimension 7. Then we define 


Q)(F.) = {V € Grin, k) | dim(V 9 Fr-x+i-a,) 2 1} 


and similarly for Q5. 


Example 3.12. The Schubert variety Qo(F.) C Gr(4, 2) consists of the two- 
dimensional subspaces V of C4 for which dim(V M Fy) > 1. Under the quotient 
map C4 —> P°, this is equivalent to space of all lines in P? that intersect a given line 
in at least a point, which is precisely the variety we need for Question 1.1. 
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3.3 A Note on Flags 


Why are chains of subspaces called flags? Roughly speaking, a flag on a flagpole 
consists of: 


A point (the top of the pole), 

A line passing through that point (the pole), 

A plane passing through that line (the plane containing the flag), and 
Space to put it in. 


Mathematically, this is the data of a complete flag in three dimensions. However, 
higher-dimensional beings would require more complicated flags. So in general, 
it is natural to define a complete flag in n-dimensional space C” to be a chain of 
vector spaces F; of each dimension from 0 to , each containing the previous, with 
dim(F;) = i for all i. A partial flag is a chain of subspaces in which only some of 
the possible dimensions are included. 


3.4 Problems 


3.1. Projective space is a Grassmannian: Show that every projective space P” is 
a Grassmannian. What are n and k? 

3.2. Schubert cells in P’”: What are the Schubert cells in P”? Express your answer 
in homogeneous coordinates. 

3.3. Schubert varieties in P”: What are the Schubert varieties in P”, thought of as a 
Grassmannian? Why are they the closures of the Schubert cells in the topology 
on P”? 

3.4. Schubert varieties versus Schubert cells: Show that every Schubert variety is 
a disjoint union of Schubert cells. Describe which Schubert cells are contained 
in &2) in terms of partitions. 

3.5. Extreme cases: Describe Qg and Qg where B is the entire ambient rectangle. 
What are their dimensions? 

3.6. Intersecting Schubert Varieties: Show that, by choosing four different flags 
FY), FO, FS), F, Question 1.1 becomes equivalent to finding the intersec- 
tion of the Schubert varieties 


Qn (FE?) 9 29 (F) 9 Qn (F2) 1 2g (FE). 


3.7. A Variety of Varieties: Translate the simple intersection problems of lines 
passing through two points, points contained in two lines, and so on into prob- 
lems about intersections of Schubert varieties, as we did for Question 1.1 in 
Problem 3.4 above. What does Question 1.2 become? 

3.8. More complicated flag conditions: In P*, let 2-planes A and B intersect in a 
point X, and let P and Q be distinct points different from X. Let S be the set 
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of all 2-planes C that contain both P and Q and intersect A and B each ina 
line. Express S' as an intersection of Schubert varieties in Gr(5, 3), in each of 
the following cases: 


(a) When P is contained in A and Q is contained in B; 
(b) When neither P nor Q lie on A or B. 


4 Variation 1: Intersections of Schubert Varieties 
in the Grassmannian 


In the previous section, we saw how to express certain linear intersection problems 
as intersections of Schubert varieties in a Grassmannian. We now will build up the 
machinery needed to obtain a combinatorial rule for computing these intersections, 
known as the Litthewood-Richardson rule. 

Both the geometric and combinatorial aspects of the Littlewood—Richardson rule 
are fairly complicated to prove, and we refer the reader to [22] for complete proofs. 
In this exposition, we will focus more on the applications and intuition behind the 
tule. 

The Littlewood-Richardson rule is particularly nice in the case of zero-dimensional 
intersections. In particular, given a list of generic flags F in C” fori =1,...,r, 
let A, ..., AM be partitions with 


So] =k — bk). 


Then the intersection 


(2. (FL) 


is zero-dimensional, consisting of exactly ong \w points of Gr(n, k), where B is 


the ambient rectangle and hee 5 is acertain Littlewood—Richardson coefficient, 
defined in Sect. 4.6. 

When we refer to a “generic” choice of flags, we mean that we are choosing from 
an open dense subset of the flag variety. This will be made more precise in Sect. 5. 

In general, the Littlewood—Richardson rule computes products of Schubert classes 
in the cohomology ring of the Grassmannian, described in Sect. 4.4 below, which cor- 
responds with (not necessarily zero-dimensional) intersections of Schubert varieties. 
To gain intuition for these intersections, we follow [22] and first simplify even further, 


to the case of two flags that intersect transversely. 


faery Av” 
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4.1 Opposite and Transverse Flags, Genericity 


Two subspaces of C” are said to be transverse if their intersection has the “expected 
dimension”. For instance, two two-dimensional subspaces of C? are expected to 
have a one-dimensional intersection; only rarely is their intersection two-dimensional 
(when the two planes coincide). More rigorously: 


Definition 4.1. Two subspaces V and W of C” are transverse if 
dim(V 9 W) = max(0, dim(V) + dim(W) — n). 
Equivalently, if codim(V) is defined to be n — dim(V), then 
codim(V NM W) = min(n, codim(V) + codim(W)). 


Exercise 4.2. Verify that the two definitions above are equivalent. 


We say two flags F{) and F) are transverse if every pair of subspaces F; and 
FO are transverse. In fact, a weaker condition suffices: 


Lemma 4.3. Two complete flags F,, E, © C" are transverse if and only if Fy~j.O 
E; = {0} for alli. 


Proof Sketch. The forward direction is clear. For the reverse implication, we can take 
the quotient of both flags by the one-dimensional subspace E and induct on n. 

Define the standard flag F, to be the flag in which F; = (e),..., e;), and similarly 
define the opposite flag E, by E; = (én, ..., €n—j+1). It is easy to check that these 
flags F, and E, are transverse. Furthermore, we shall see that every pair of transverse 
flags can be mapped to this pair, as follows. Consider the action of GL, (C) on C” 
by standard matrix multiplication, and note that this gives rise to an action on flags 
and subspaces, and subsequently on Schubert varieties as well. 


Lemma 4.4. For any pair of transverse flags F and E’,, there is an element g € GLn 
such that gF! = F, and gE’ = E,, where F, and E, are the standard and opposite 
flags. 


The proof of this lemma is left as an exercise to the reader (see the Problems 
section below). The important corollary is that to understand the intersection of 
the Schubert varieties Q)(F{) and Q,,(E‘), it suffices to compute the intersection 
Q) CF.) A Q) CE.) and multiply the results by the appropriate matrix g. 

So, when we consider the intersection of two Schubert varieties with respect to 
transverse flags, it suffices to consider the standard and opposite flags F. and E,. 
We use this principle in the duality theorem below, which tells us exactly when the 
intersection of Q2)(F,) and Q,,(£,) is nonempty. 
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4.2 Duality Theorem 


Definition 4.5. Two partitions \ = (\i,..., Ax) and ps = (f41, ..., Wx) are comple- 
mentary in the k x (n — k) ambient rectangle if and only if A; + uei-i =n —k 
for all 7. In this case we write ° = X. 


In other words, if we rotate the Young diagram of ju and place it in the lower right 
corner of the ambient rectangle, its complement is \. Below, we see that pp = (3, 2) 
is the complement of \ = (4, 2, 1) in Gr(7, 3). 


Theorem 4.6 (Duality Theorem). Let F, and E, be transverse flags in C", and 
let X and «us be partitions with |A| + |u| = k(n — k). In Gr(n, k), the intersection 
Q) (FL) O Q,CE.) has 1 element if js and \ are complementary partitions, and is 
empty otherwise. Furthermore, if 4 and X are any partitions with [ig4,—; + Ai > 
n —k for some i then Q)(F.) 0 Q,(E.) = Y. 


We can use a reversed form of row reduction to express the Schubert varieties 
with respect to the opposite flag, and then the Schubert cells for the complementary 
partitions will have their 1’s in the same positions, as in the example below. Their 
unique intersection will be precisely this matrix of 1’s with 0’s elsewhere. 


00 0 0 0 0 1 x 0 0 1 
0 0 0 1 « «x +O *x O 1 0 
Oo 1 *« O «x *« O * 1 0 0 


ox * 
oo * 
oo x 


We now give a more rigorous proof below, which follows that in [22] but with a 
few notational differences. 


Proof. We prove the second claim first: if for some i we have pip4)—; + A; > n —k 
then Q)(F.) 1 Q,,(E.) is empty. Assume for contradiction that there is a subspace 
V in the intersection. We know dim(V) = k, and also 
dim(V 1M (e1, €2,.--, €n—K+i-d;)) = i, (1) 
dim(V 9 (én, @n—15 «+ +s Cnt1—(n—k+ k4 1-1) —pess_i))) ZK +1 i. 
Simplifying the last subscript above, and reversing the order of the generators, we 
get 
dim(V 9 (ip jypgy ps ++ +s Cn—-1, Cn) 2>K+1—i. (2) 


Notice that i + pgy1-; > n —k +i — 2; by the condition pyyi_; + A; > nN —k, 
so the two subspaces we are intersecting with V in Eqs. (1) and (2) are disjoint. It 
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follows that V has dimension at least k + 1 —i +i =k+ 1, a contradiction. Thus, 
Q) (FL) ON Q,,(E.) is empty in this case. 

Thus, if |A| + |u| = &(@ — k) and \ and yz are not complementary, then the inter- 
section is empty as well, since the inequality uy4i—-; + A; > n — k must hold for 
some /. 

Finally, suppose A and 41 are complementary. Then Eqs.(1) and (2) still hold, 
but now n —k+i— A; =i + uny1-i for all i. Thus dim(V A (ei+,,,,_;)) = 1 for 
alli = 1,...,, and since V is k-dimensional it must equal the span of these basis 
elements, namely 

V= (E14 pins C24 bn-po tte Ce pina t—e)* 


This is the unique solution. 


Example 4.7. We now can give a rather high-powered proof that there is a unique 
line passing through any two distinct points in P”. As before, we work in one higher 
dimensional affine space and consider 2-planes in C’*'. Working in Gr(n + 1, 2), 
the two distinct points become two distinct one-dimensional subspaces F; and EF 
of C”*!, and the Schubert condition that demands the two-dimensional subspace V 
contains them is 

dim(V M Fi) > 1, dim(V MN E;) > 1. 


These are the Schubert conditions for a single-part partition \ = (A,) where (n + 
1)-—2+1—-—A,; =1. Thus \; =n — 1, and we are intersecting the Schubert vari- 
eties 

Qa-1) Fe) N Qar—1) (Ee) 


where F, and E, are any two transverse flags extending F, and E,. Notice that (n — 1) 
and (n — 1) are complementary partitions in the 2 x (n — 1) ambient rectangle (see 
Fig. 2), so by the Duality Theorem there is a unique point of Gr(m + 1, 2) in the 
intersection. The conclusion follows. 


4.3, Cell Complex Structure 


In order to prove the more general zero-dimensional Littlewood—Richardson rule and 
compute the Littlewood—Richardson coefficients, we need to develop more heavy 
machinery. In particular, we need to understand the Grassmannian as a geometric 
object and compute its cohomology, an associated ring in which multiplication of 


Fig. 2. Two complimentary 
partitions of size n — | filling Lo 
the n — 1 x 2 rectangle 
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certain generators will correspond to intersection of Schubert varieties. (See [27] for 
more details on all of the material in this section.) 

The term Schubert cell comes from the notion of a cell complex (also known as a 
CW complex) in algebraic topology. An n-cell is a topological space homeomorphic 
to the open ball |v| < 1 in R’, and its associated n-disk is its closure |v| < 1 in R”. 

To construct a cell complex, one starts with a set of points called the 0-skeleton 
X°, and then attaches 1-disks D via continuous boundary maps from the boundary 
OD (which consists of two points) to X °. The result is a 1-skeleton X!. 

This can then be extended to a 2-skeleton by attaching 2-disks D via maps from 
the boundary 0D (which is a circle) to X!. In general, the n-skeleton X” is formed 
by attaching a set of n-disks to X"~! along their boundaries. 

More precisely, to form X” from X "—1 we start with a collection of n-disks Dr 
and continuous attaching maps y, : 0D” — X"~'. Then 


a Les 


~ 


x" 


where ~ is the identification x ~ y a(x) for x € 0D”. The cell complex is X = 
U,, Xx”, which may be simply X = X” if the process stops at stage n. By the con- 
struction, the points of X° along with the open i-cells associated with the i-disks in 
X! for each i are disjoint and cover the cell complex X. The topology is given by 
the rule that A C X is open if and only if AM X” is open in X” for all n, where the 
topology on X" is given by the usual Euclidean topology on R”. 


Example 4.8. The real projective plane Pe has a cell complex structure in which 
X°={(0:0: D}isa single point, X' = X°u{(0:1:*)} is topologically a circle 
formed by attaching a 1-cell to the point at both ends, and then X? is formed by 
attaching a 2-cell IR? to the circle such that the boundary wraps around the 1-cell 
twice. This is because the points of the form (1 : xt : yt) ast + oo andast — —oo 
both approach the same point in X!, so the boundary map must be a 2-to-1 mapping. 


Example 4.9. The complex projective plane Pe has a simpler cell complex structure, 
consisting of starting with a single point X° = {(0: 0: 1)}, and then attaching a 2- 
cell (a copy of C = R’) like a balloon to form X*. A copy of C? = R’ is then attached 
to form X*. 


The Schubert cells give a cell complex structure on the Grassmannian. For a 
complete proof of this, see [53], section 3.2. We sketch the construction below. 

Define the 0-skeleton X° to be the zero-dimensional Schubert variety Q (n—1)4)- 
Define X7 to be X° along with the 2-cell (since we are working over C and not R) 
given by Qn k-1y? and the attaching map given by the closure in Gr(n, k). 
Note that the partition in this step is formed by removing a single corner square from 
the ambient rectangle. 

Then, X* is formed by attaching the two four-cells given by removing two 


: : 4 ‘ : 
outer corner squares in both possible ways, giving either nb) n—k—Ln—k-1) or 
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nye! n—k—-2) We can continue in this manner with each partition size to define 


the entire cell structure, X° C X2. C++» Cc X7kM-#), 


Example 4.10. We have 


Gr (4, 2) = QF 5 U O75 UY O75 QF U QF UG, 


° 


forming a cell complex structure in which X° = Q2_, X? is formed by attaching 


Q°_, X* is formed by attaching 22 Li 22, X° is formed by attaching Q2, and X® 


is formed by attaching 925. 


4.4 Cellular Homology and Cohomology 


For a CW complex X = X° C --- C X", define 
G = tk-cells 


the free abelian group generated by the k-cells B® = (D)?. 
Define the cellular boundary map di+) : Cr+; — Cx by 


dy (BE*?) = ) deg, s+ BS”, 
B 


where deg, , is the degree of the composite map 
OBE)  xF Be 


The first map above is the cellular attaching map from the boundary of the closure of 
the ball B“*” to the k-skeleton, and the second map is the quotient map formed by 
collapsing X* \ B es to a point. The composite is a map from a k-sphere to another 
k-sphere, which has a degree, whose precise definition we omit here and refer the 
reader to [27], section 2.2, p. 134. As one example, the 2-to- 1 attaching map described 
in Example 4.8 for Pe has degree 2. 

It is known that the cellular boundary maps make the groups C,; into a chain 
complex: a sequence of maps 


dy dn-1 qd 
O—> Cy > Ch-1 — Ch-2 > ++: > Ci @ Co > 0 


for which d; o dj+; = 0 for all i. This latter condition implies that the image of the 


map d;+1 1s contained in the kernel of d; for all i, and so we can consider the quotient 
groups 
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A(X) = ker(d;)/im(di+1) 


for all i. These quotients are abelian groups called the cellular homology groups of 
the space X. 


Example 4.11. Recall that Pe consists of a point, a 2-cell, and a 4-cell. So, its 
cellular chain complex is: 


>0-07-07-Z2-0-2Z2-0-2Z2-0 


and the homology groups are Hy) = H) = Hy = Z, H,; = H;3 = 0. 
On the other hand, in P2, the chain complex looks like: 


Oo-Z-Z2-Z-0 


where the first map Z — Z is multiplication by 2 and the second is the 0 map, due to 
the degrees of the attaching maps. It follows that Hy = 0, H; = Z/2Z, and Hp = Z. 


We can now define the cellular cohomology by dualizing the chain complex 
above. In particular, define 


c* = Hom(C,, Z) = {group homomorphisms f : Cy, > Z} 
for each k, and define the coboundary maps d; : C‘~! > C* by 
di f (c) = f(d(c)) 


for any f € C* and c € Cy. Then the coboundary maps form a cochain complex, 
and we can define the cohomology groups to be the abelian groups 


H'(X) = ker(d*,,)/im(d}) 


for all i. 


Example 4.12. The cellular cochain complex for PY is 


0-Z-0-2Z2-07-2Z2-0-0-0-.::-: 


and so the cohomology groups are H® = H* = H*+=Z, H' = H? =0. 


Finally, the direct sum of the cohomology groups 
H*(X) = @ Hi(X) 


has a ring structure given by the cup product ((27], p. 249), which is the dual of 
the “cap product” ([27], p. 239) on homology and roughly corresponds to taking 
intersection of cohomology classes in this setting. 
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In particular, there is an equivalent definition of cohomology on the Grassmannian 
known as the Chow ring, in which cohomology classes in H*(X) are equivalence 
classes of algebraic subvarieties under birational equivalence. (See [21], Sections 
1.1 and 19.1.) In other words, deformations under rational families are still equivalent: 
in P’, for instance, the family of algebraic subvarieties of the form xy — tz? = 0 
as t € C varies are all in one equivalence class, even as t > 0 and the hyperbola 
degenerates into two lines. 

The main fact we will be using under this interpretation is the following, which 
we state without proof. (See [22], Section 9.4 for more details.) 


Theorem 4.13. The cohomology ring H*(Gr(n,k)) has a Z-basis given by the 
classes 
0 = [Q)(F.)] € HP (Grin, &)) 


for 4 a partition fitting inside the ambient rectangle. The cohomology H*(Gr(n, k)) 
is a graded ring, 800-0, € H?)\+2\Hl (Gr(n, k)), and we have 


OV: On = [92, (Fe) a Q,,(E.)] 


where F, and E, are the standard and opposite flags. 


Note that a) is independent of the choice of flag F., since any two Schubert 
varieties of the same partition shape are rationally equivalent via a change of basis. 
We can now restate the intersection problems in terms of multiplying Schubert 
classes. In particular, if A, ..., A” are partitions with )°; |\| = k(n — k), then 


yw -++oyn € H*’— (Grn, k)) 


and there is only one generator of the top cohomology group, namely og where B 
is the ambient rectangle. This is the cohomology class of the single point Q;(F,) 
for some flag F,. Thus the intersection of the Schubert varieties Qa (FS DAN 
Q)y» (F) is rationally equivalent to a finite union of points, the number of which 


B 


For a sufficiently general choice of flags, the cya) yw 


distinct with no multiplicity. 
In general, we wish to understand the coefficients that we get upon multiplying 
Schubert classes and expressing the product back in the basis {a7} of Schubert classes. 


points in the intersection are 


Example 4.14. In Problem 3.4, we saw that Question 1.1 can be rephrased as com- 
puting the size of the intersection 


20 (FL?) N 2a (F) 9 On(FY) 0 Qg(F) 
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for a given generic choice of flags F{”,..., F. By the above analysis, we can 
further reduce this problem to computing the coefficient c for which 


00:00:00:00=Cc:9¢ 


in H*(Gr(4, 2)). 


4.5 Connection with Symmetric Functions 


We can model the cohomology ring H*(Gr(n, k)) algebraically as a quotient of the 
ring of symmetric functions. We only cover the essentials of symmetric function 
theory for our purposes here, and refer the reader to Chapter 7 of [51], or the books 
[38] or [48] for more details, or to [22] for the connection between H*(Gr(n, k)) 
and the ring of symmetric functions. 


Definition 4.15. The ring of symmetric functions Ac(x,X2,...) is the ring of 
bounded-degree formal power series f € C[[x1, x2, ...]] which are symmetric under 
permuting the variables, that is, 


f (1, X2,-..) = f(%na), Xray, +++) 


for any permutation 7 : Z, —> Z4 and deg(f) < oo. 


For instance, x? + x} + x} +--+ is a symmetric function of degree 2. 

The most important symmetric functions for Schubert calculus are the Schur 
functions. They can be defined in many equivalent ways, from being characters of 
irreducible representations of S, to an expression as a ratio of determinants. We use 
the combinatorial definition here and start by introducing some common terminology 
involving Young tableaux and partitions. 


Definition 4.16. A skew shape is the difference v/\ formed by cutting out the 
Young diagram of a partition A from the strictly larger partition v. A skew shape is 
a horizontal strip if no column contains more than one box. 


Definition 4.17. A semistandard Young tableau (SSYT) of shape v/\ is a way 
of filling the boxes of the Young diagram of v/ with positive integers so that the 
numbers are weakly increasing across rows and strictly increasing down columns. 
An SSYT has content j if there are j; boxes labeled i for each i. The reading word 
of the tableau is the word formed by concatenating the rows from bottom to top. 


The following is a semistandard Young tableau of shape v/ and content j1 where 
vy = (6,5, 3), A = (3, 2), and px = (4, 2, 2, 1). Its reading word is 134223111. 


Ue) A 
PAs, 


i) 
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Definition 4.18. Let \ be a partition. Given a semistandard Young tableau T of 


my m2 


shape \, define x7 = xj"'x5""--- where m; is the number of i’s in 7. The Schur 
function for a partition \ is the symmetric function defined by 


5S), = ) x? 
a 


where the sum ranges over all SSYT’s T of shape 4. 


Example 4.19. For \ = (2, 1), the tableaux 


1/1 1/2 1/1 1/2 
2 2 3 3 2 


— 
Ww 


are a few of the infinitely many SSYT’s of shape \. Thus we have 
ae 2 2 
S\ = XpX2 + XyxX_ AX X3 + 2xpx2x3 +--+: . 


It is well known that the Schur functions s) are symmetric and form a vector space 
basis of A(x,, x2, ...) as A ranges over all partitions. The key fact that we will need 
is that they allow us to understand the cohomology ring H*(Gr(n, k)), as follows. 


Theorem 4.20. There is a ring isomorphism 
H* (Gr(n, k)) = A(xq, X2,--)/(5,|A Z B) 


where B is the ambient rectangle and (s,|A ¢ B) is the ideal generated by the Schur 
functions. The isomorphism sends the Schubert class 0 to the Schur function s). 


This is a pivotal theorem in the study of the Grassmannian, since it allows us to 
compute in the cohomology ring simply by working with symmetric polynomials. 
In particular, multiplying Schur functions corresponds to multiplying cohomology 
classes, which in turn gives us information about intersections of Schubert varieties. 

As an approach to prove Theorem 4.20, note that sending a) to s) is an iso- 
morphism of the underlying vector spaces, since on the right-hand side we have 
quotiented by the Schur functions whose partition does not fit inside the ambient 
rectangle. So, it remains to show that this isomorphism respects the multiplications 
in these rings, taking cup product to polynomial multiplication. 

An important first step is the Pieri Rule. For Schur functions, this tells us how to 
multiply a one-row shape by any other partition: 


Sir) S\ = > Sis 


v/X horz. strip of size r 


We wish to show that the same relation holds for the 7,’s, that is, that 


www.TechnicalBooksPDF.com 


Variations on a Theme of Schubert Calculus 137 


Or) (OX = ¥, Ov, 


v/X horz. strip of size r 


where the sum on the right is restricted to partitions v fitting inside the ambient 
rectangle. Note that we do not need this restriction for general Schur functions, but 
in the cohomology ring we are considering the quotient by partitions not fitting inside 
the ambient rectangle, so the two expansions above are not exactly the same. 

Note that, by the Duality Theorem, we can multiply both sides of the above relation 
by o,.< to extract the coefficient of o,, on the right-hand side. So, the Pieri Rule is 
equivalent to the following restatement: 


Theorem 4.21 (Pieri Rule). Let X and yu be partitions with |X| + |u| = k(n — k) — 
r. Then if F., E,, and H, are three sufficiently general flags then the intersection 


Q) (Fe) al Q,,(E.) Nn Qo) CA.) 


has \ element if p°/X is a horizontal strip, and it is empty otherwise. 


Sketch of Proof: We can set F, and E, to be the standard and opposite flags and H, 
a generic flag distinct from F, or E,. We can then perform a direct analysis similar 
to that in the Duality Theorem. See [22] for full details. O 

Algebraically, the Pieri rule suffices to show the ring isomorphism, because the 
Schur functions s;,) and corresponding Schubert classes o,,) form an algebraic set 
of generators for their respective rings. Therefore, to intersect Schubert classes we 
simply have to understand how to multiply Schur functions. 


4.6 The Littlewood—-Richardson Rule 


The combinatorial rule for multiplying Schur functions, or Schubert classes, is called 
the Litthewood—Richardson Rule. To state it, we need to introduce a few new 
notions. 


Definition 4.22. A word wyjw2---w, (where each w; € {1,2,3,...}) is 
Yamanouchi (or lattice or ballot) if every suffix w,w,z41--- w, contains at least 
as many letters equal toi asi + 1 for alli. 


For instance, the word 231211 is Yamanouchi, because the suffixes 1, 11, 211, 
1211, 31211, and 231211 each contain at least as many 1’s as 2’s, and at least as 
many 2’s as 3’s. 


Definition 4.23. A Littlewood—Richardson tableau is a semistandard Young 
tableau whose reading word is Yamanouchi. 


Exercise 4.24. The example tableau in Fig. 3 is not Littlewood—Richardson. Why? 
Can you find a tableau of that shape that is? 
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Fig. 3. An example of a T]111 

skew Littlewood—Richardson 

tableau 2/2/3 
1/3/4 


Definition 4.25. A sequence of skew tableaux 7, 7, ... forma chain if their shapes 
do not overlap and 
T; UT U-:-UT; 


is a partition shape for all i. 


We can now state the general Littlewood—Richardson rule. We will refer the reader 
to [22] for a proof, as the combinatorics is quite involved. 


Theorem 4.26. We have 


Vv 


where CX) is the number of chains of Littlewood—Richardson tableaux of con- 


oe jor 
tents \“) with total shape v. 


It is worth noting that in many texts, the following corollary is the primary focus, 
since the above theorem can be easily derived from the m = 2 case stated below. 


Corollary 4.27. We have 


Vv 
SS = ) Cy Sv 
Vv 


where Cha is the number of Littlewood—Richardson tableaux of skew shape v/X and 
content |. 


Proof. By Theorem 4.26, c\ ,, 18 the number of chains of two Littlewood-Richardson 
tableaux of content \ and yw with total shape v. The first tableau of content is 
a straight shape tableau, so by the Yamanouchi reading word condition and the 
semistandard condition, the top row can only contain 1’s. Continuing this reasoning 
inductively, it has only 7’s in its ith row for each i. Therefore, the first tableau in the 
chain is the unique tableau of shape \ and content A. 

Thus, the second tableau is a Littkewood—Richardson tableau of shape v/A and 
content js, and the result follows. 


As a consequence of Theorems 4.26 and 4.20, in H*(Gr(n, k)) we have 


OVO) eres O\m) = ) Xa Many Fv 
V 
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where now the sum on the right is restricted to partitions v fitting inside the ambient 
rectangle. Note that by the combinatorics of the Littlewood—Richardson rule, the 
coefficients on the right are nonzero only if |v| = > |A|, and so in the case of a 
zero-dimensional intersection of Schubert varieties, the only possible v on the right- 
hand side is the ambient rectangle B itself. Moreover, Q2,(F.) is a single point of 
Gr(n, k) for any flag F,. The zero-dimensional Littlewood—Richardson rule follows 
as a corollary. 


Theorem 4.28 (Zero-Dimensional Littlewood—Richardson Rule). Let B be the k x 
(n — k) ambient rectangle, and let \,..., 1" be partitions fitting inside B such 
that |B| = 9°; |A® |. Also let FO, ..., F&” be any m generic flags. Then 


is equal to the number of chains of Littlewood—Richardson tableaux of contents 
AD, ..., A” with total shape equal to B. 


Example 4.29. Suppose k =3 and n—k=4. Let A? = (2,1), A? = (2,1), 
\®) = (3,1), and A“ = 2. Then there are five different chains of Littlewood— 
Richardson tableaux of contents \,..., AM that fill the k x (n — k) ambient rect- 


freer 


Example 4.30. We can now solve Question 1.1. In Example 4.14, we showed that 
it suffices to compute the coefficient c in the expansion 


00°90:'90:00 =c:on 


in H*(Gr(4, 2)). This is the Littkewood—Richardson coefficient ce . This is 
the number of ways to fill a 2 x 2 ambient rectangle with a chain of Littlewood— 
Richardson tableaux having one box each. 

Since such a tableau can only have a single | as its entry, we will label the entries 
with subscripts indicating the step in the chain to distinguish them. We have two 
possibilities, as shown in Fig.5. Therefore the coefficient is 2, and so there are 2 
lines passing through four generic lines in P*. 


In Example 4.30, we are in the special case in which each Littlewood—Richardson 
tableau in the chain has only one box, and so the only choice we have is the ordering 


Fig. 4 The five chains of Littlewood—Richardson tableaux of contents AD = (2, 1), N® = (2, 1), 
A®) = (3, 1), and A® = 2 filling an ambient 3 x 4 rectangle 
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Fig. 5 The two tableaux chains used to enumerate the Littlewood—Richardson coefficient that 
answers Question 1.1 


of the boxes in a way that forms a chain. We can, therefore, simply represent such a 
tableau by its indices instead, and the two tableaux of Fig. 5 become 


1/2 1|3 
214 and 241, 


The two tableaux above are characterized by the property that the entries 1, 2, 3, 4 
are used exactly once and the rows and columns are strictly increasing. Such a tableau 
is called a standard Young tableaux. 


Definition 4.31. A standard Young tableau of shape \ with |A| = 7 is an SSYT 
of shape A in which the numbers 1, 2, ..., are each used exactly once. 


There is a well-known explicit formula, known as the Hook lIength formula, for 
the number of standard Young tableaux of a given shape, due to Frame, Robinson, 
and Thrall [18]. To state it we need the following definition. 


Definition 4.32. For a square s in a Young diagram, define the hook length 
hook(s) = arm(s) + leg(s) + 1 


where arm(s) is the number of squares strictly to the right of s in its row and leg(s) 
is the number of squares strictly below s in its column. 


Theorem 4.33 (Hook length formula.) The number of standard Young tableaux of 
shape X is 
|A|! 
Tle, hook(s) ° 
For example, if \ = (2, 2) then we have oa = 2 standard Young tableaux of 
shape A, which matches our answer in Example 4.30. 


4.7 Problems 


4.1. Prove Lemma 4.4: For any transverse flags F{ and E/, there is some g € GL, 
such that gF{ = F, and gE’ = E,, where F, and E, are the standard and 
opposite flags. 
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4.2. It’s all Littlkewood—Richardson: Verify that the Duality Theorem and the Pieri 
Rule are both special cases of the Litthewood—Richardson rule. 
4.3. An empty intersection: Show that 


Qa, (Fe) ON Qa (E.) =o 


in Gr(4, 2) for transverse flags F, and E,. What does this mean geometrically? 
4.4. A nonempty intersection: Show that 


Qa (Fe) A Qe (Ee) 


is nonempty in Gr(5, 2). (Hint: intersecting it with a certain third Schubert 
variety will be nonempty by the Littlewood—Richardson rule.) What does this 
mean geometrically? 

4.5. Problem 3.4 revisited: In P*, suppose the 2-planes A and B intersect in a point 
X,and P and Q are distinct points different from X. Show that there is exactly 
one plane C that contains both P and Q and intersect A and B each in a line as 
an intersection of Schubert varieties in Gr(5, 3), in each of the following cases: 


(a) When P is contained in A and Q is contained in B; 
(b) When neither P nor Q lie on A or B. 


4.6. That’s a lot of k-planes: Solve Question 1.2 for a generic choice of flags as 
follows. 


(a) Verify that the problem boils down to computing the coefficient of s(¢,_;)«) 
in the product of Schur functions 8, 


(b) Use the Hook Length Formula to finish the computation. 


5 Variation 2: The Flag Variety 


For the content in this section, we refer to [39], unless otherwise noted below. 

The (complete) flag variety (in dimension 7) is the set of all complete flags in C”, 
with a Schubert cell decomposition similar to that of the Grassmannian. In particular, 
given a flag 

WiivoVUc::-yW=aC', 


we can choose n vectors vj, ..., UV, Such that the span of v;,..., v; is V; for each i, 
and list the vectors uv; as row vectors of ann x n matrix. We can then perform certain 
row reduction operations to form a different ordered basis Ui. ..., U), that also span 


the subspaces of the flag, but whose matrix entries consist of a permutation matrix 
of 1’s, all 0’s to the left and below each 1, and arbitrary complex numbers in all other 
entries. 
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For instance, say we start with the flag in three dimensions generated by the 
vectors (0, 2,3), (1, 1, 4), and (1, 2, —3). The corresponding matrix is 


0 2 3 
1 1 4 
L. 2. 3 
We start by finding the leftmost nonzero element in the first row and scale that row 


so that this element is 1. Then subtract multiples of this row from the rows below it 
so that all the entries below that 1 are 0. Continue the process on all further rows: 


0 2 3 O 1 15 0 1 15 
11 4 ]7> 71 0 25)7> 71 0 25 
1 2 -3 1 0 -6 0 0 1 


It is easy to see that this process does not change the flag formed by the initial row 
spans and that any two matrices in canonical form define different flags. So, the flag 
variety is a cell complex consisting of n! Schubert cells indexed by permutations. 
For instance, one such open set in the five-dimensional flag variety is the open set 
given by all matrices of the form 


O Ll *« * x 
1 O «x « x 
0 0 0 0 1 
0 0 1 x*« O 
0 0 0 1 +0 


We call this cell Xj<5,,, because 4,5, 1,3, 2 are the positions of the 1’s from the 
right-hand side of the matrix in order from top to bottom. More rigorously, we define 
a Schubert cell as follows. 


Definition 5.1 Let w € S,, be a permutation of {1, ...,}. Then the Schubert cell 
of w is defined by 


X), ={V. € Fl, : dim(V, 9 F,) = #{i < p: w(i) < q} forall p, g} 


where F, is the standard flag generated by the unit vectors e,,;_;. In the matrix form 
above, the columns are ordered from right to left as before. 


Note that, as in the case of the Grassmannian, we can choose a different flag F, 
with respect to which we define our Schubert cell decomposition, and we define 
Xs, (F.) accordingly. 

The dimension of a Schubert cell X,, is the number of *’s in its matrix, that is, the 
number of entries above and right of the pivot | in its row and column. The maximum 
number of *’s occurs when the permutation is wo = n(n — 1)---321, in which case 
the dimension of the open set X,,, is n(n — 1)/2 (or n(n — 1) over R). In general, 
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it is not hard to see that the number of *’s in the set X,, is the inversion number 
inv(w). This is defined to be the number of pairs of entries (w(i), w(j)) of w which 
are out of order, that is, i < j and w(i) > w(/). Thus we have 


dim(X7,) = inv(w). 


Example 5.2 The permutation w = 45132 has seven inversions. (Can you find them 
all?) We also see that dim(X},) = 7, since there are seven * entries in the matrix. 


Another useful way to think of inv(w) is in terms of its length. 


Definition 5.3 Defines), ..., 5,-, € S, tobe the adjacent transpositions in the sym- 
metric group, that is, s; is the permutation interchanging i andi + 1. Then the length 
of w, written £(w), is the smallest number k for which there exists a decomposition 


W = Sj, +++ Si,- 


Lemma 5.4 We have ¢(w) = inv(w) for any w € Sy. 


We will leave the proof of this lemma as an exercise to the reader in the Problems 
Section. 


5.1 Schubert Varieties and the Bruhat Order 


By using the Pliicker embeddings Gr(n, k) — PG! for each k, we can embed 
Fl, into the larger projective space P””~' whose entries correspond to the Pliicker 
coordinates of each of the initial k x n submatrices of a given element of the flag 
variety. This makes Fl, a projective subvariety of P?’~! (see [22] for more details), 
which in turn gives rise to a topology on Fl, known as the Zariski topology. Now, 
consider the closures of the sets Xj, in this topology. 


Definition 5.5 The Schubert variety corresponding to a permutation w € S,, is 
aes oe 


As in the Grassmannian, these Schubert varieties turn out to be disjoint unions 
of Schubert cells. The partial ordering in which X,, = U,<yX> is called the Bruhat 
order, a well-known partial order on permutations. We will briefly review it here, 
but we refer to [10] for an excellent introduction to Bruhat order. 


Definition 5.6 The Bruhat order < on S,, is defined by v < w if and only if, for 
every representation of w as a product of /(w) transpositions s;, one can remove 
l(w) — l(v) of the transpositions to obtain a representation of v as a subword in the 
same relative order. 
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Example 5.7 The permutation w = 45132 can be written as 5253525)545352. This 
contains 535253 = 14325 as a (non-consecutive) subword, and so 14325 < 45132. 


5.2 Intersections and Duality 


Now suppose we wish to answer incidence questions about our flags: which flags 
satisfy certain linear constraints? As in the case of the Grassmannian, this boils down 
to understanding how the Schubert varieties X,, intersect. 

We start with the Duality Theorem for Fl,,. Following [22], it will be convenient 
to define dual Schubert varieties as follows. 


Definition 5.8 Let E, be the standard and opposite flags, and for shorthand we let 
Xy = X(F.) and 
Yy = X wo-w( Ee) 


where wo = n(n — 1)---1 is the longest word. The set Y,, is often called a dual 
Schubert variety. 


Notice that 
dim(Y,,) = inv(wo - w) = n(n — 1)/2 — inv(w) 


since if w’ = wo - w then w’(i) =n + 1 — w(i) for all i. 


Theorem 5.9 (Duality Theorem, V2.). /f/(w) = l(v), we have Xy OY, =Gifw x 
vand |X» Y,| = 1 ifw = v. Furthermore, if l(w) < l(v) then Xy OY, = &. 


The proof works similarly to the Duality Theorem in the Grassmannian. In par- 
ticular, with respect to the standard basis, the dual Schubert variety Y,, is formed by 
the same permutation matrix of 1’s as in X,,, but with the 0 entries below and to the 
right of the 1’s (and x entries elsewhere). For instance, we have 


O 1 * «x x * 1 0 0 0 
1 0 *« * x 1 0 0 0 0 
X 45132 = 0 0 0 0 1 ; Y45132 = 0 0 * x 1 
0 0 1 x*« O 0 0 1 0 0 
0 0 0 1 =O 0 0 0 1 =O 


and their intersection contains only the permutation matrix determined by w = 
45132. 
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5.3. Schubert Polynomials and the Cohomology Ring 


In order to continue our variation on the theme, it would be natural at this point to 
look for a Pieri rule or a Littlewood—Richardson rule. But just as the cohomology 
ring of the Grassmannian and the Schur functions made those rules more natural, we 
now turn to Schubert polynomials and the cohomology ring H*(FI,,) over Z. 

This ring has a natural interpretation as a quotient of a polynomial ring. In 
particular, letting a, be the cohomology class of Y,,, we have o, € H 21 (Fl,) 
where i = inv(w). For the transpositions s;, we have o;, € H ?(Fl,,). The elements 
Xj = Os, — O5,,, fori <n — landx, = —o,,_, gives a set of generators for the coho- 
mology ring, and in fact 


AY (Fn) = Zl, ..., Xn)/(e1, --- Cn) =: Ra 


where €),..., @, are the elementary symmetric polynomials in x), ..., X,. (See [22] 
or [6].) 

The ring R, is known as the coinvariant ring and arises in many geometric and 
combinatorial contexts. Often defined over a field k rather than Z, its dimension as 
a k-vector space (or rank as a Z-module) is n!. There are many natural bases for R, 
of size n!, such as the monomial basis given by 

{xf'-+-x 2a; <n—i for alli} 
(see, for instance [23]), the harmonic polynomial basis (see [5], Section 8.4) and 
the Schubert basis described below. There are also many famous generalizations 
of the coinvariant ring, such as the Garsia—Procesi modules [23] and the diagonal 
coinvariants (see [5], Chapter 10), which are closely tied to the study of Macdonald 
polynomials in symmetric function theory [38]. 

The Schubert polynomials form a basis of R, whose product corresponds to the 
intersection of Schubert varieties. To define them, we require a divided difference 
operator. 


Definition 5.10. For any polynomial P(x,,...,x,) € Z[x1, ..., 2X], we define 
P—5;(P 
a,(P) = Pan) 
Xi — Xi+1 
where s;(P) = P(x1,..., Xj-1, Xi41, Xi, Xi42, +++, Xn) is the polynomial formed by 


switching x; and x;,; in P. 
We can use these operators to recursively define the Schubert polynomials. 


Definition 5.11. We define the Schubert polynomials G,, for w € S, by: 


e Sy, = ime me ee ae where wo = n(n — 1)---21 is the longest permu- 
tation, 
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e Ifw ¢ wo, finda minimal factoriation of the form w = wo - Sj, - ++ Sj, isaminimal 
factorization of its form, that is, a factorization for which (wo - 5;, + +++ 5;,) = 
n— p forall 1 < p<r.Then 


6, = Oi, fe) hed Oo eee Oi, (Sw) 


Remark 5.12. One can show that the operators 0; satisfy the two relations below. 


e Commutation Relation: 0,0; = 0,0; for any i, j with |i — j| > 1, 
e Braid Relation: 0; O;410; = 034.10; O41 for any 1. 


Since these (along with 5? = 1) generate all relations satisfied by the reflections 
s; (see chapter 3 of [10]), the construction in Definition 5.11 is independent of the 
choice of minimal factorization. Note also that a? = 0, so the requirement of minimal 
factorizations is necessary in the definition. 


The Schubert polynomials’ image in R,, not only form a basis of these cohomology 
rings, but the polynomials themselves form a basis of all polynomials in the following 
sense. The Schubert polynomials G,, are well defined for permutations w € S. = 
LU) Sin for which w(i) > w(i + 1) for all i > k for some k. For a fixed such k, these 
Schubert polynomials form a basis for Z[x1, ..., x¢]. 

One special case of the analog of the Pieri rule for Schubert polynomials is known 
as Monk’s rule. 


Theorem 5.13 (Monk’s rule). We have 
6,<6,= 6, 


where the sum ranges over all permutations v obtained from w by: 


e Choosing a pair p,q of indices with p <i < q for which w(p) < w(q) and for 
any k between p and q, w(k) is not between w(p) and w(q), 

e Defining v(p) = w(q), v(q) = w(:p) and for all other k, v(k) = w(k). 

Equivalently, the sum is over all v = w-t where t is a transposition (pq) with 


Dp <i <q forwhich lv) =1(w) + 1. 


Interestingly, there is not a known “Littlewood—Richardson rule” that generalizes 
Monk’s rule, and this is an important open problem in Schubert calculus. 


Open Problem 5.14. Find a combinatorial interpretation analogous to the 
Littlewood-Richardson rule for the positive integer coefficients cj, in the expan- 
sion 


S6,=) of Gu 


and therefore for computing the intersection of Schubert varieties in Fly. 


Similar open problems exist for other partial flag varieties, defined in the next 
sections. 
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5.4 Two Alternative Definitions 


There are two other ways of defining the flag manifold that are somewhat less explicit 
but more generalizable. The group GL, = GL,(C) acts on the set of flags by left 
multiplication on its ordered basis. Under this action, the stabilizer of the standard 
flag F, is the subgroup B consisting of all invertible upper triangular matrices. Notice 
that GL, acts transitively on flags via change-of-basis matrices, and so the stabilizer 
of any arbitrary flag is simply a conjugation gBg~! of B. We can, therefore, define 
the flag variety as the set of cosets in the quotient GL, /B, and define its variety 
structure accordingly. 

Alternatively, we can associate to each coset gB in GL, /B the subgroup gBg"!. 
Since B is its own normalizer in G (gBg~! = B iff g € B), the cosets in GL, /B 
are in one-to-one correspondence with subgroups conjugate to B. We can, therefore, 
define the flag variety as the set 6 of all subgroups conjugate to B. 


5.5 Generalized Flag Varieties 


The notion of a “flag variety” can be extended in an algebraic way starting from the 
definition as GL, /B, to quotients of other matrix groups G by certain subgroups 
B called Borel subgroups. The subgroup B of invertible upper triangular matrices 
is an example of a Borel subgroup of GL,, that is, a maximal connected solvable 
subgroup. It is connected because it is the product of the torus (C*)” and (3) copies of 
C. We can also show that it is solvable, meaning that its derived series of commutators 


Bo := B, 
B, := [Bo, Bol, 
By := [Bi, Bi), 


terminates. Indeed, [B, B] is the set of all matrices of the form bcb~!c7! for b andc in 
B. Writing b = (d; + ,) and c = (d; +2) where d, and d) are diagonal matrices 
and n; and ny strictly upper triangular, it is not hard to show that bcb~'c7! has all 
1’s on the diagonal. By a similar argument, one can show that the elements of By 
have 1’s on the diagonal and 0’s on the off-diagonal, and B3 has two off-diagonal 
rows of 0’s, and so on. Thus, the derived series is eventually the trivial group. 

In fact, a well-known theorem of Lie and Kolchin [31] states that all solvable 
subgroups of GL, consist of upper triangular matrices in some basis. This implies that 
B is maximal as well among solvable subgroups. Therefore, B is a Borel subgroup. 

The Lie—Kolchin theorem also implies that all the Borel subgroups in GL, are 
of the form gBg~! (and all such groups are Borel subgroups). That is, all Borel 
subgroups are conjugate. It turns out that this is true for any semisimple linear 
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algebraic group G, that is, a matrix group defined by polynomial equations in the 
matrix entries, such that G has no nontrivial smooth connected solvable normal 
subgroups. 

Additionally, any Borel subgroup in a semisimple linear algebraic group G is its 
own normalizer. By an argument identical to that in the previous section, it follows 
that the groups G/B are independent of the choice of Borel subgroup B (up to 
isomorphism) and are also isomorphic to the set B of all Borel subgroups of G as 
well. Therefore, we can think of B as an algebraic variety by inheriting the structure 
from G/B for any Borel subgroup B. 

Finally, we can define a generalized flag variety as follows. 


Definition 5.15. The flag variety of a semisimple linear algebraic group G to be 
G/B where B is a Borel subgroup. 


Some classical examples of such linear algebraic groups are the special linear 
group SL, the special orthogonal group SO,, of orthogonal n x n matrices, and the 
symplectic group S P2, of symplectic matrices. We will explore a related quotient of 
the special orthogonal group S$ O2,+4; in Sect. 6. 

We now define partial flag varieties, another generalization of the complete flag 
variety. Recall that a partial flag is a sequence F;, C --- C Fj, of subspaces of C” 
with dim(F;,) = i; for all j. Notice that a k-dimensional subspace of C” can be 
thought of as a partial flag consisting of a single subspace Fy. 

It is not hard to show that all partial flag varieties, the varieties of partial flags 
of certain degrees, can be defined as a quotient G/P for a parabolic subgroup P, 
namely a closed intermediate subgroup B C P C G. The Grassmannian Gr(n, k), 
then, can be thought of as the quotient of GL,, by the parabolic subgroup S = Stab(V) 
where V is any fixed k-dimensional subspace of C”. Similarly, we can start with a 
different algebraic group, say the special orthogonal group SO>,41, and quotient by 
parabolic subgroups to get partial flag varieties of other types. 


5.6 Problems 


5.1. Reflection length equals inversion number: Show that /(w) = inv(w) for any 
we Sy. 

5.2. Practice makes perfect: Write out all the Schubert polynomials for permuta- 
tions in $3 and Sy. 

5.3. Braid relations: Verify that the operators 0; satisfy the braid relations as stated 
in Remark 5.12. 

5.4. The product rule for Schubert calculus: Prove that 0;(P - Q) = 0;(P)-Q+ 
s;(P) - 0;(Q) for any two polynomials P and Q. 

5.5. Divided difference acts on R,,: Use the previous problem to show that the 
operator O; maps the ideal generated by elementary symmetric polynomials to 
itself, and hence, the operator descends to a map on the quotient R,,. 
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5.6. Schubert polynomials as a basis: Prove that if w € Sx satisfies w(i) > w(i + 
1) for alli > k then G, € Z[x,..., x,]. Show that they form a basis of the 
polynomial ring as well. 


6 Variation 3: The Orthogonal Grassmannian 


In the previous section, we saw that we can interpret the Grassmannian as a partial flag 
variety. We can generalize this construction to other matrix groups G, hence, defining 
Grassmannians in other Lie types. We will explore one of these Grassmannians as 
our final variation. 


Definition 6.1. The orthogonal Grassmannian OG(2n + 1,k) is the quotient 
SO2,+1/P where P is the stabilizer of a fixed isotropic k-dimensional subspace V. 
The term isotropic means that V satisfies (v, w) = 0 for all v, w € V with respect 
to a chosen symmetric bilinear form (, ). 


The isotropic condition, at first glance, seems very unnatural. After all, how could 
a nonzero subspace possibly be orthogonal to itself? Well, it is first important to 
note that we are working over C, not R, and the bilinear form is symmetric, not 
conjugate-symmetric. In particular, suppose we define the bilinear form to be the 
usual dot product 


((Qy,-- +, Gong), (D1, «+ «5 Bangi) = aby + gba + +++ + Gon41 donq 


in C2"+!, Then in C?, the vector (3, 5i, 4) is orthogonal to itself: 3-3-+57-5i+4- 
4=0. 

While the choice of symmetric bilinear form does not change the fundamental 
geometry of the orthogonal Grassmannian, one choice in particular makes things 
easier to work with in practice: the “reverse dot product” given by 


2n+1 
(a1, +65 Gangs Diy -6+ Dongt)) = D> aiDong1—i- 


i=1 


In particular, with respect to this symmetric form, the standard complete flag F, is an 
orthogonal flag, with F} = F»,+1-; for alli. Orthogonal flags are precisely the type 
of flags that are used to define Schubert varieties in the orthogonal Grassmannian. 

Note that isotropic subspaces are sent to other isotropic subspaces under the action 
of the orthogonal group: if (v, w) = 0 then (Av, Aw) = (v, w) = 0 for any A € 
SO2n41. Thus the orthogonal Grassmannian OG(2n + 1, k), which is the quotient 
SOo,+1/Stab(V), can be interpreted as the variety of all k-dimensional isotropic 
subspaces of C?7"+1, 
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6.1 Schubert Varieties and Row Reduction in OG(2n + 1, n) 


Just as in the ordinary Grassmannian, there is a Schubert cell decomposition for 
the orthogonal Grassmannian. The combinatorics of Schubert varieties are particu- 
larly nice in the case of OG(2n + 1, n) in which the orthogonal subspaces are “half 
dimension” n. (See the introduction of [54] or the book [26] for more details.) 

In Gr(2n + 1, n), the Schubert varieties are indexed by partitions \ whose Young 
diagram fit inside the n x (n + 1) ambient rectangle. Suppose we divide this rect- 
angle into two staircases as shown below using the blue cut, and only consider the 
partitions \ that are symmetric with respect to the reflective map taking the upper 
staircase to the lower. 


* 

* | ok 

* | kK YK | 
% | K | K | Ok Yk 


We claim that the Schubert varieties of the orthogonal Grassmannian are indexed 
by the shifted partitions formed by ignoring the lower half of these symmetric 
partition diagrams. We define the ambient triangle to be the half of the ambient 
rectangle above the staircase cut. 


Definition 6.2. A shifted partition is a strictly decreasing sequence of positive 
integers, \ = (Ay >... > Ax). We write |\| = > \;. The shifted Young diagram 
of 4 is the partial grid in which the ith row contains \; boxes and is shifted to the 
right i steps. Below is the shifted Young diagram of the shifted partition (3, 1), drawn 
inside the ambient triangle from the example above. 


Definition 6.3. Let F, be an orthogonal flag in C””*!, and let be a shifted partition. 
Then the Schubert variety X(F.) is defined by 


X\(F.) = {W € OG2n + 1,n) : dim(WN F,444;- 


7) >ifori=1,...,n} 
where } is the “doubled partition” formed by reflecting the shifted partition about 
the staircase. 


In other words, the Schubert varieties consist of the isotropic elements of the 
ordinary Schubert varieties, giving a natural embedding OG(2n + 1,n) — Gr(2n + 
1, n) that respects the Schubert decompositions: 
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X\(F.) = Q5(F.) NOG (2n + 1, n). 


To get a sense of how this works, consider the example of \ = (3, 1) and A= 
(4, 3, 1) shown above, in the case n = 4. The Schubert cell Qe in Gr(9, 4) looks like 


| = S| oS 
+ Sor=> 
oF SS 
*¥ */|O;}O 
ee) —) || — 
aman y | — | =) 
* * *|O 
coo oF 
x* * * * 


Now, which of these spaces are isotropic? Suppose we label the starred entries as 
shown, omitting the 0 entries: 


Qahs. 
or. a ™~™ 
hwWN 


1fg 
1a boc 
We will show that the entries /, j,k, h,i,e are all uniquely determined by the 
values of the remaining variables a, b, c, d, f, g. Thus, there is one isotropic subspace 


in this cell for each choice of values a, b, c,d, f, g, corresponding to the “lower half” 
of the partition diagram we started with, namely 


l 
alk 
fi 9 thli 
alb|\cidite 


To see this, let the rows of the matrix be labeled 1, 2,3, 4 from top to bottom as 
shown, and suppose its row span is isotropic. Since row 1 and 4 are orthogonal with 
respect to the reverse dot product, we get the relation 


l+a=0, 


which expresses / = —a in terms of a. 
Rows 2 and 4 are also orthogonal, which means that 


b+k=0, 


so we can similarly eliminate k. From rows 2 and 3, we obtain f + j = 0, which 
expresses j in terms of the lower variables. We then pair row 3 with itself to see that 
h + g* = 0, eliminating h, and finally pairing 3 with 4 we have i + gc +d = 0, so 
i is now expressed in terms of lower variables as well. 
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Moreover, these are the only relations we get from the isotropic condition—any 
other pairings of rows give the trivial relation 0 = 0. So in this case the Schubert 
variety restricted to the orthogonal Grassmannian has half the dimension of the 
original, generated by the possible values for a, b,c, d, f, g. 


6.2 General Elimination Argument 


Why does the elimination process work for any symmetric shape A? Label the steps 
of the boundary path of \ by 1, 2,3, ... from SW to NE in the lower left half, and 
label them from NE to SW in the upper right half, as shown: 


Then the labels on the vertical steps in the lower left half give the column indices of 
the 1’s in the corresponding rows of the matrix. The labels on the horizontal steps in 
the upper half, which match these labels by symmetry, give the column indices from 
the right of the corresponding starred columns from right to left. 


ae) 5 
| 1 
1 * x 
1 * x * Ox 
1 x : ae * Ox 
1 * Ox * Ox 
—_ 5 3 2 


This means that the 1’s in the lower left of the matrix correspond to the opposite 
columns of those containing letters in the upper right half. It follows that we can use 
the orthogonality relations to pair a 1 (which is leftmost in its row) with a column 
entry in a higher or equal row so as to express that entry in terms of other letters 
to its lower left. The | is in a lower or equal row in these pairings precisely for the 
entries whose corresponding square lies above the staircase cut. Thus, we can always 
express the upper right variables in terms of the lower left, as in our example above. 
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Fig. 6 The tableau above is 171 [2" 
a shifted semistandard 

tableau of shape \/j. where 1'}2 

A = (6,4, 2, 1) and 1}1 

ft = (3, 2), and content 

(5, 2, 1). Its reading word is 3 
3111/21/12’ 


6.3 Shifted Tableaux and a Littlewood-Richardson Rule 


The beauty of shifted partitions is that so much of the original tableaux combinatorics 
that goes into ordinary Schubert calculus works almost the same way for shifted 
tableaux and the orthogonal Grassmannian. We define these notions rigorously below. 


Definition 6.4. A shifted semistandard Young tableau is a filling of the boxes of a 
shifted skew shape with entries from the alphabet {1 < 1 < 2! <2 <3’ <3 <---} 
such that the entries are weakly increasing down columns and across rows, and such 
that primed entries can only repeat in columns, and unprimed only in rows. 

The reading word of such a tableau is the word formed by concatenating the 
rows from bottom to top. The content of T is the vector content(T) = (71, 12, ...), 
where n; is the total number of (i)s and (i’)s in T. See Fig. 6 for an example. 


In this setting, there are actually two analogs of “Schur functions” that arise from 
these semistandard tableaux. They are known as the Schur P-functions and Schur 
Q-functions. 


Definition 6.5. Let A/, be a shifted skew shape. Define ShST g(A/,1) to be the set 
of all shifted semistandard tableaux of shape A/j1. Define ShST p(A/,) to be the set 
of those tableaux in which primes are not allowed on the staircase diagonal. 


Definition 6.6. The Schur Q-function Q),,, is defined as 


Oyj, Mev) = > w(L) 


TeShST9(A/p) 


and the Schur P-function P),,, is defined as 


T 
Pi Rie. = Ss wer). 
T eShST p(A/y1) 


The Schur Q-functions, like ordinary Schur functions, are symmetric functions 
with unique leading terms, spanning a proper subspace of A. In addition, they have 
positive product expansions 


154 M. Gillespie 
£ £(v)—L(A) ¢r 
0,0 n= yO Oy 


for certain positive integers iis It is easy to see that this is equivalent to the rule 


P,Py = > page 


Here the coefficients ie are precisely the structure coefficients for the cohomology 


ring of the orthogonal Grassmannian. In particular, if we extend them to generalized 
coefficients by 


Py ao ee Py = ‘ Fv. Py, 
we have the following theorem due to Pragacz [44]. 


Theorem 6.7. A zero-dimensional intersection X,1---OX, has exactly 
FUO...pf0 points, where T is the ambient triangle. 


Stembridge [52] first found a Littlewood—Richardson-type rule to enumerate these 
coefficients. The rule is as follows. 


Definition 6.8. Let T be a semistandard shifted skew tableau with the first i or 


i’ in reading order unprimed, and with reading word w = w,---+ wy. Let m;(j) be 
the multiplicity of i among w,_j+1,..., Wn (the last j entries) for any i and for 
any j <n. Also let p;(j) be the multiplicity of i’ among w),..., w;. Then T is 


Littlewood—-Richardson if and only if 


e Whenever m;(j) = mi4i1(j) we have w,_; Ai +1, G+ 1)’, and 
e Whenever m;(n) + pi(j) = mi4i(n) + pi(j) we have wj41 Ai, G+ 1)’. 


Notice that this definition implies that m;(j) > m;+1(j) for all i and j, which 
is similar to the usual Litthewood—Richardson definition for ordinary tableaux. An 
alternative rule that only requires reading through the word once (rather than once 
in each direction, as in the definition of m; above) is given in [25]. 


6.4 Problems 


6.1. Show that, if \ is a partition that is not symmetric about the staircase cut, the 
intersection Q5 (F.) 1 OG(2n + 1, n) is empty. 

6.2. How many isotropic 3-planes in C’ intersect six given 3-planes each in at least 
dimension |? 
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7 Conclusion and Further Variations 


In this exposition, we have only explored the basics of the cohomology of the Grass- 
mannian, the complete flag variety, and the orthogonal Grassmannian. There are 
many other natural directions one might explore from here. 

First and foremost, we recommend that interested readers next turn to Fulton’s 
book entitled Young Tableaux [22] for more details on the combinatorial aspects of 
Schubert calculus and symmetric functions, including connections with representa- 
tion theory. Other books that are a natural next step from this exposition are those 
of Manivel [39], Kumar on Kac—Moody groups and their flag varieties [33], and 
Billey—Lakshmibai on smoothness and singular loci of Schubert varieties [9]. 

In some more specialized directions, the flag varieties and Grassmannians in 
other Lie types (as briefly defined in Sect.6) have been studied extensively. The 
combinatorics of general Schubert polynomials for other Lie types was developed by 
Billey and Haiman in [8] and also by Fomin and Kirillov in type B [20]. Combinatorial 
methods for minuscule and cominuscule types is presented in [54]. 

It is also natural to investigate partial flag varieties between the Grassmannian and 
FI,,. Buch, Kresch, Purbhoo, and Tamvakis established a Litthewood—Richardson rule 
in the special case of two-step flag varieties (consisting of the partial flags having just 
two subspaces) in [13], and the three-step case was very recently solved by Knutson 
and Zinn-Justin [30]. Coskun provided a potential alternative approach in terms of 
Mondrian tableaux, with a full preliminary answer for partial flag varieties in [15], 
and for the two-row case in [16]. 

Other variants of cohomology, such as equivariant cohomology and K -theory, 
have been extensively explored for the Grassmannian and the flag variety as well. 
An excellent introduction to equivariant cohomology can be found in [2, 12] is a 
foundational paper on the K-theory of Grassmannians. The K-theoretic analog of 
Schubert polynomials are called Grothendieck polynomials, first defined by Lascoux 
and Schutzenberger [36]. 

Another cohomological variant is quantum cohomology, originally arising in 
string theory and put on mathematical foundations in the 1990s (see [32, 46]). Fomin, 
Gelfand, and Postnikov [19] studied a quantum analog of Schubert polynomials and 
their combinatorics. Chen studied quantum cohomology on flag manifolds in [14], 
and the case of equivariant quantum cohomology has been more recently explored 
by Anderson and Chen in [3] and Bertiger, Mili¢evi¢, and Taipale in [7]. In [41, 
42], Pechenik and Yong prove a conjecture of Knutson and Vakil that gives a rule 
for equivariant K-theory of Grassmannians. The list goes on; there are many coho- 
mology theories (in fact, infinitely many, in some sense) all of which give slightly 
different insight into the workings of Grassmannians and flag varieties. 

It is worth noting that Young tableaux are not the only combinatorial objects that 
can be used to describe these cohomology theories. Knutson, Tao, and Woodward 
developed the theory of puzzles in [29], another such combinatorial object which 
often arises in the generalizations listed above. 
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On the geometric side, Vakil [56] discovered a “geometric Littlewood—Richardson 
Rule” that describes an explicit way to degenerate an intersection of Schubert varieties 
into a union of other Schubert varieties (not just at the level of cohomology). This, in 
some sense, more explicitly answers the intersection problems described in Sect. 1. 

Another natural geometric question is the smoothness and singularities of Schu- 
bert varieties. Besides the book by Billey and Lakshmibai mentioned above [9], this 
has been studied for the full flag variety by Lakshmibai and Sandya [35], in which 
they found a pattern avoidance criterion on permutations w for which the Schubert 
variety X,, is smooth. Related results on smoothness in partial flag varieties and other 
variants have been studied by Gasharov and Reiner [24], Ryan [47], and Wolper [57]. 
Abe and Billey [1] summarized much of this work with a number of results on pattern 
avoidance in Schubert varieties. 

Real Schubert calculus (involving intersection problems in real n-dimensional 
space IR”) is somewhat more complicated than the complex setting, but there are 
still many nice results in this area. For instance, a theorem of Mukhin, Tarasov, and 
Varchenko in [40] states that for a choice of flags that are each maximally tangent at 
some real point on the rational normal curve, the intersections of the corresponding 
complex Schubert varieties have all real solutions. An excellent recent overview of 
this area was written by Sottile in [50]. 

Relatedly, one can study the positive real points of the Grassmannian, that is, 
the subset of the Grassmannian whose Pliicker coordinates have all positive (or 
nonnegative) real values. Perhaps one of the most exciting recent developments is 
the connection with scattering amplitudes in quantum physics, leading to the notion of 
an amplituhedron coming from a positive Grassmannian. An accessible introduction 
to the main ideas can be found in [11], and for more in-depth study, the book [4] by 
Arkani-Hamed et al. In [43], Postnikov, Speyer, and Williams explore much of the 
rich combinatorial foundations of the positive Grassmannian. 

Finally, there are also many geometric spaces that have some similarities with 
the theory of Grassmannians and flag varieties. Hessenberg varieties are a family of 
subvarieties of the full flag variety determined by stability conditions under a chosen 
linear transformations (see Tymoczko’s thesis [55], for instance). Lee studied the 
combinatorics of the affine flag variety in detail in [37]. The book k-Schur functions 
and affine Schubert calculus by Lam, Lapointe, Morse, Schilling, Shimozono, and 
Zabrocki [34] gives an excellent overview of this area, its connections to k-Schur 
functions, and the unresolved conjectures on their combinatorics. 
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Combinatorics of the Diagonal Mm) 
Harmonics ona 


Angela Hicks 


Abstract The Shuffle Theorem, recently proven by Carlsson and Mellit, states that 
the bigraded Frobenius characteristic of the diagonal harmonics is equal to a weighted 
sum of parking functions. In this introduction to the topic, we discuss the theorem and 
connections between it and the well-known Macdonald polynomials. Furthermore, 
we describe important combinatorial bijections which imply various restatements of 
the theorem and play an important role in its proof. Finally, we briefly discuss the 
proof and describe various generalizations of the theorem. 


The diagonal harmonics are a simply defined vector space of polynomials in the 
variables X;, = {x1,--* ,Xn} and ¥, = {y,--+ , Yn}: 


Definition (Diagonal harmonics). 


DHy = {f (Xn, Yn) € C[Xn, Yu): ye O, Oy, f (Xn, Yn) = 0 forallr,s => 0, r+s > 0}, 


i=l" 


with a natural G,, action which permutes the X,, and Y,, variables simultaneously by 
acting on their indices. As a vector space, DH, has proven to be a nontrivial puzzle to 
mathematicians, with questions like “What is its dimension?” inspiring new tools in 
algebraic combinatorics. Haiman [33] proved, using tools from algebraic geometry, 
that 

dim(DH,,) = (n+ 1)°"?. 


This suggested that perhaps one could supplement complex calculations about DH, 
by finding a concrete family of combinatorial objects of size (n + 1)~) which 
can be used as a combinatorial model for algebraic properties of DH,,. The Shuffle 
Theorem, recently proven by Carlsson and Mellit [12], does exactly this, tying the 
combinatorics of parking functions to DH, in a precise way. 


A. Hicks (BX) 
Lehigh University, Bethlehem, PA, USA 
e-mail: anh316 @lehigh.edu 


© The Author(s) and the Association for Women in Mathematics 2019 159 
H. Barcelo et al. (eds.), Recent Trends in Algebraic Combinatorics, 

Association for Women in Mathematics Series 16, 

https://doi.org/10.1007/978-3-030-05141-9_5 


160 A. Hicks 


In what follows, we begin with a brief discussion of parking functions and their 
early appearance in the literature, followed by additional information on Macdonald 
polynomials meant to give context for the Shuffle Theorem and why it has been of 
historic interest. In Sect.2, we carefully state the Shuffle Theorem and review the 
relevant definitions required to understand it. In Sect.3, we describe from several 
perspectives a couple of important maps on the parking functions which lead to 
combinatorial restatements of the Shuffle Theorem. These maps, which occur at 
key points in the proof of the Shuffle Theorem and its generalization, have several 
dramatically different characterizations within the literature, and its our hope that by 
giving them and their various descriptions in one place, we’ll make the connection 
between the older and newer literature more accessible. In Sect.4, we give some 
necessarily vague (limited by our abbreviated text) indication of the final proof of 
the Shuffle Theorem. Finally, in Sect.5 we give information about how the Shuffle 
Theorem has been generalized. We would be remiss if we did not immediately 
mention two well-written texts in this area which cover most of these topics and 
many more in much greater detail; both Haglund [28] and Bergeron [7] are excellent 
starting places for additional details, although they were written before the proof of 
the Shuffle Theorem was completed. 


1 Background 


Parking functions show up in a number of guises, from their initial inception as an 
idealized data storage method popular in theoretical computer science, to probability 
where their various statistics have ties to surprising distributions (see Yan [59] for 
a summary), and to algebraic combinatorics (our primary interest), where they are 
used to explore the representation theory of the diagonal harmonics via the Shuffle 
Theorem. 

In their simplest form, parking functions are a subset of (Z/nZ)"; here we follow 
the convention that we use {1, 2,--- ,} (but not 0) for the names of the equivalence 
classes. Think of 

= (Ty soy Tr) 


as giving the preferences of a list of drivers, so that the ith driver wants to park in 
space 7;. The drivers proceed (one by one) to park in n spaces, with the first driver 
parking in his preferred space 7,. The ith driver does the same, unless his preferred 
space 7; is already occupied by a previous driver, in which case he proceeds to the 
next open space. If we imagine that there are exactly n spaces, 7 is a parking function 
if and only if every driver is able to park. We call the set of parking functions on n 
spaces PF,,. 

It’s easy to see that if any two drivers want to park in space n, 7 will fail to be 
a parking function since only n — 2 drivers will consider parking in the first n — 1 
spaces, and thus one such space will be empty. Expanding the idea slightly, it must 
be that if 7 is in PF,,, forall 1 <j <n, 
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Him < j}>j. 


Remarkably this is a necessary and sufficient condition for 7 € (Z/nZ)" to be a 
parking function, and one often sees parking functions defined this way directly: 


Definition (Parking Function). 7 = (7, 72, +++ , 7) in (Z/nZ)" is a parking func- 
tion (Le., is in PF,,) if for 1 < j <n, #{i: 7; < j} > j. 


Thus, for example, (5,2, 1, 4, 2) is a parking function, but (5, 2, 1, 5, 2) is not. 


1.1 Parking Functions in Enumerative Combinatorics 


Pyke [47] was the first to define parking functions, using somewhat different moti- 
vation and language than our definition above. The language of cars on a one way 
street is due to Konheim and Weiss [38] who described them while working for 
IBM Research. Their colorful (if not politically correct) story of “dutiful” husbands 
driving down a one way street until their “cantankerous” wives order them to park 
was the beginning of an analysis of a basic hashing technique, where information is 
stored on a disk at a specified location, unless that location is already in use. Com- 
puter science literature refers to this as “linear probing in a successful search” (as, 
for example, in Knuth [37]), and in this language, there is a lot known about the 
enumeration of parking functions according to various statistics. Pollak (as relayed 
by Foata and Riordan [13]) gives the best-known proof of the size of PF,,: 


Theorem 1. |PF,,| = (n+ 1)""!. 


Proof. Consider a street with n + 1 spaces (so 7 is in (Z/(n + 1)Z)"), where those 
spaces are set out along a circular street, so that if the ith driver finds parking spots 7; 
through spots + | occupied, he circles back to the beginning of the street to continue 
his hunt at the first spot. In this scenario, every 7 “parks” and since there is one 
more space than the number of cars, we have exactly one space that remains empty. 
Moreover, if we consider the resulting sequence of parked cars when we start with the 
preference 7 and when we start with the preference 7 + (1,..., 1) € (Z/(n + 1)Z)’, 
the latter is a rotation of the former by exactly one space (with the empty space rotating 
similarly). Thus for any 7, 


{r+tk(,..., Dike Z/(n+ WZ} 


can be thought of as an equivalence class of functions whose resulting parked cars 
are the same up to rotation; it is clearly size (n + 1). Moreover, 7 in this setup 
is a parking function if and only if when drivers park according to 7, the empty 
parking space is at n + 1. Thus the parking functions make particularly nice repre- 
sentatives of each class and there must be ch = (n+ 1)""! parking functions as 
desired. 
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A number of other families of objects are in bijection with parking functions: the 
OEIS entry for a, = (n + 1)"~! includes, in addition to parking functions, spanning 
trees on the labeled complete graph on n vertices, the number of edge-labeled rooted 
trees on n nodes, the number of ways of expressing an n cycle as an a product of 
n — | transpositions, as well as a number of others [54]. 


1.2. Symmetric Function Background 


Parking functions play an interesting role in symmetric function theory and G,, rep- 
resentation theory. Statistics on parking functions historically led to the development 
of a combinatorial formula for one of the important families of symmetric function 
bases: the Macdonald polynomials. 

In the following, we use A” for the vector space generated by homogeneous 
symmetric functions of degree n over a field K, {m)})-, for the monomial basis, 
{e,}\-n for the elementary basis, {1)}\-, for the homogeneous basis, {p\},-n for 
the power sum basis, and {s)})-+, for the Schur basis. Stanley [55] and Sagan [51] 
contain definitions of all these bases for the unfamiliar reader. 

We use <I for dominance order on partitions, so that \ < if for alli, Ay +--+ 
Ai XS py + +++ + py. If we use m;(A) for the multiplicity of 7 in A (.e., mj(A) = #{7 : 
A; = i}), then recall the Hall inner product can be defined by: 


(Pr; Pu) = Zrlyg=p, 


where zy = [], i”m;(A)! and 1x is the standard indicator function that is 1 if X 
is true and O if X is false. With these definitions in hand, then recall that one can 
uniquely define the Schur functions {s)})-, by their following properties: 


(1) (Orthonormality.) (5), s,,) = Ta,- 
(2) (Triangularity.) s, = panes K)y,,m) where K),, = 1. 


Recall that Schur function expansions are important in our understanding of the 
image of the Frobenius characteristic map. For a given class function ¢ of G, recall 
that 


1 
Fehar(g) := = > ep OH) Pu 
is 


The irreducible representations of G, are indexed by partitions, and the Frobenius 
map takes such an irreducible representation to a Schur function indexed by the same 
partition. Thus every G,, module has an associated Schur positive polynomial, and in 
reverse, when a polynomial is Schur positive, it is reasonable to search for a natural 
associated G, module. Sagan [51] gives a nice introduction to these facts for the 
unfamiliar reader. 
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1.3, Macdonald Polynomials 


Beyond the above five bases for A”, a basis of particular importance is 
{P\[X; q, t]},\-n, which we now call the Macdonald polynomials. (We now work 
in K[q, t].) Macdonald [44] originally defined them as a q, t analogue of the Schur 
basis. In particular, he began with a qg, t analogue of the Hall inner product: 


(l—q*) 
(Pr Pugs = 2] | ado) 


Then P,[X; q, t] is uniquely defined by: 


(1) (Orthogonality.) (P\[X; q,t], PulX; ¢@. t))¢4 = calyap. 
(2) (Triangularity.) 


PAX; 4g. t= ody nq. ty, 
war 


where dj \(qg,t) = 1. 


Replacing t or g in P,[X; q, t] with various integer values (usually 0, 1, or a limit) 
gives a number of famous symmetric function bases, from the Schur functions them- 
selves (at f = gq = 0) and monomials (f = 1) to the Hall—Littlewood, the Jack, the 
Askey—Wilson, and the Koornwinder polynomials. In short, Macdonald polynomials 
encode information about most of the important bases of the symmetric functions. 

Although P,[X; q, t] is referred to as a Macdonald polynomial, strictly speak- 
ing it is a symmetric function, with coefficients which are rational functions in g 
and t; Macdonald also defined J,[X; g, t] = c\(q, t) P\[X; ¢, t] which is a different 
rescaled version of P,[X;q, tf] using a combinatorially defined constant c)(q, t), 
chosen such that J,[X; q, t] has coefficients which are polynomial in qg and ft. (See 
Macdonald [44, 8.1].) Macdonald’s Schur Positivity Conjecture (see Macdonald [44, 
8.18]) stated that J,\[X; q, t] was “almost” Schur positive: expressing J,[X; qg, t] in 
terms of a closely related t-analogue of the Schurs {s,,(x; t)},+n gave coefficients 
that are polynomials in g and f with positive integral coefficients. More precisely, 
{s,LX; t]} yen differs from {s,,},+n by a simple homomorphism: let ¢ be the multi- 
plicative homomorphism that for all nonnegative integers, k sends the power sum px 
to px — t*). Then 

SLX; t] = O(s,) = s, [X00 — 1], 


where the last term gives the same expression using plethystic notation. See Loehr 
and Remmel [43] for complete details on the notation. 

Garsia suggested several different manipulations of the Macdonald polynomials, 
applying the inverse homomorphism (and flipping the powers of t— sending t > | 
and multiplying by a power of t) defining two new families of polynomials. 
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xX 
Ay [X;q,t]:= Jy | ——3q,t 
1-t 
and 7 
By (keg. eH Pw ay, [ey 1/0); 


where n(ju) is just the highest power of ¢ in H) [X; g, t]. All the above-mentioned 
families of polynomials from the P,[X; ¢, t] to the H \LX; ¢g, t] are referred to by 
various authors as “the Macdonald polynomials,” although their chosen notation— 
using P, J or H —generally differentiates them. 

The modified Macdonald polynomials are in fact Schur positive, as Macdonald’s 
Schur Positivity Conjecture. Garsia and Haiman [17] gave a combinatorially defined 
polynomial A for each partition of X of n. In particular, for a Ferrer’s diagram of the 
partition \ drawn in French notation, with its southwest corner at the origin, Garsia 
and Haiman take the set {(p1, g1),--- , (Dns Gn)} Of southwest corners of each cell 
in the diagram. (Thus in particular, (0, 0) will correspond to the bottom left cell.) 
Then, they define 

Ay = det [lx7" ¥7" Hijet n- 

See Fig. 1 for an example when \ = (3, 2). Garsia and Haiman then considered the 
linear span of all the partial derivatives of the Ay, which we will denote £L[O, 0, A)]. 
Moreover, they showed that if for all A, the dimension of £[0, OyA \] is n!, then the 
Frobenius image of £[0,0,A,] is H \LX; q, t], and thus, the modified Macdonald 
polynomials are Schur positive. The statement about the dimension is the well-known 
n! Conjecture, which was proved almost a decade later in Haiman [32]. His proof 
thus implied that the coefficient of g't/ in Hy[X; q, t] is the Frobenius character of 
the subspace of £[0,0, A] of total degree i in the x variables and j in the y variables, 
and moreover, that Macdonald’s Schur Positivity Conjecture is true. While Haglund 
et al. [30] gives a combinatorial formula (as opposed to an indirect definition via 
triangularities) for the modified Macdonald polynomials, there is great interest in a 
general formula the modified Macdonald—Kostka coefficients 


Kuxq, tis (A [X; q; t], 51) : 


At the time of publication, the most complete results in this direction are presented 
in Assaf [4]. 


2 The Diagonal Harmonics 


A closely related space to £[0, 0, A,] is the diagonal harmonics; recall that as men- 
tioned above: 


Definition (Diagonal harmonics). 
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Aso =det | |2gy3 ©3y$ 23y3 <3y3 2343 


Fig. 1 Computing A3 2 


DHn = {f (Xn, Yn) € C[Xn, Yu): s G05, Ff (Xn, Yn) = 0 for ally, s > 0, r+s > O}. 


i=l 
Theorem 2. [fj n, then L[OxOyA,,] is a subspace of DHn. 


Proof. Let M,, be the matrix whose determinant is A,,, and let (M )'7 denote the 


(i, j)th minor of M,,. Let aia = aug Bo Tad An is clearly O if (a, b) is nota cell 
which occurs in 4, So we assume (a, b) = (7, qi) for some / in {1,--- , n}. 


y ORT : Sy eet 
= i Ji Wi,j=l,---.n 
oF OK (PI; "= (pr qn) ete 


q1) 
= 3 ie Ye pitsp yf (M,)"* 
fal (Pt WW) 4a oer 


n n 


0 ae 
aD a (ac at’) my" 


j=l k=l 


For a fixed j, the final summation is also the determinant of a matrix calculated 
using expansion by minors, in particular M,, with the jth column replaced by the 
given partial derivatives. Moreover, the jth column is either identically 0 or a scalar 
multiple of another column in M,, corresponding to a cell in the Ferrer’s diagram of 
Lt which is south and west. Thus for every j, the determinant is 0. 


Haiman [33] computed the dimension of both DH, and L[0,0, A] using similar 
techniques from algebraic geometry, in particular employing the Hilbert scheme of 
points in the plane, using a method of approach first sketched by Procesi years before. 
As will be discussed in more detail below, eigenoperators of the modified Macdonald 
polynomials play a key role in the symmetric function theory used to study the larger 
space of diagonal harmonics. 

The bigraded Frobenius characteristic of the diagonal harmonics has had a con- 
jectured combinatorial formula since the 1990s (before any combinatorial formula 
was suggested for the Macdonald polynomials), although the proof wasn’t completed 
until two decades later. Let DH“? give the submodule of DH, that is total degree 
a in the ‘x’ variables and b in the ‘y’ variables. Then we can define the bigraded 
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Frobenius characteristic as 


DH, LX; q,t] = )°t%q? Fchar DH?”. 
a,b 


The Shuffle Theorem, first conjectured by Haglund et al. [30], was proved recently 
by Carlsson and Mellit [12], building on the previous work of Haiman [33]: 


Theorem 3 (The Shuffle Theorem). 


DH, [X; q; t] = > gn ga) Figes(r)- (1) 
TEPF,, 


In the following, we’ll discuss the definitions of area, dinv, and ides. Fs is Gessel’s 
fundamental quasisymmetric function. Fundamental quasisymmetric functions are 
often represented with the subscript S a composition of n, but as elsewhere in the 
literature on parking functions, it will be simpler to assume the subscript S is a subset 
of {1,2,--- ,2-— 1} and not a composition of n. There is, of course, a standard 
bijection between the two, which can be employed to switch from one convention to 
the other. 

It’s worth noting that that even in the literature written after its proof, Theorem 3 
is still often referred to as “The Shuffle Conjecture.” More details are contained in 
Sect. 4 about its proof. 

Researchers routinely picture parking functions more diagrammatically to com- 
pute the statistics appearing in the right side of the theorem. As first suggested by 
Adriano Garsia, parking functions are viewed in ann x n lattice with: 


(1) A Dyck path—north and east steps from the bottom left to top right which stay 
(weakly) above the line x = y. 

(2) Labels (or “cars”)—the numbers | to 1, each occurring exactly once to the right 
of a north step, so that they are increasing from bottom to top within a column. 


To see that this definition matches our first definition, starting from such a diagram, 
define 7 = (7, --- , 7) such that 7; = j if a j is in the ith column of the lattice 
diagram. The Dyck path condition is equivalent to the rule that we must have at least 
k cars less than or equal to k for all 1 < k <n. The increasing column condition 
defines a fixed ordering of the cars within a given column, which all had the same 
preferred spot. (Without this or a similar an ordering, the map would not be bijective.) 
See Fig. 2 for an example. 

Starting with a parking function represented in a lattice, we then have the following 
definition. 


Definition (area). The area of a parking function 7 is the number of complete squares 
between the Dyck path of 7 and the main diagonal and will be denoted area(z). 


Example 1. The area of the parking function in Fig. 2 is (adding it by row) 0 + 0+ 
14+24+1+04+1+0+0=5. 
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Fig. 2 A lattice diagram of 
the parking function 

(8,9, 2,6, 2,4, 1,2, 6) in 
PFo as drawn in a lattice 
diagram 


ECE 


We refer to a diagonal of the parking function as the cells along a line y = x + b 
for some 0 < b <n. 


Definition (dinv). A primary diagonal inversion is a pair of cars in the same diagonal, 
with the smaller further southwest. A secondary diagonal inversion is a pair of cars 
in two adjacent diagonals, with the smaller car in the lower diagonal (with lower 
y-intercept), but northeast of the larger car in the higher diagonal. The dinv of a 
parking function 7 is the number of primary and secondary diagonal inversions and 
will be denoted dinv(7). 


Example 2. In Fig.2, the pair {5,9} is a primary diagonal inversion, but {1, 4} is 
not a primary inversion since the larger car (4) is further southwest. Similarly, {4, 5} 
makes a secondary inversion, but {8, 9} does not (because of the relative positions of 
the smaller and larger car) nor would any choice of car number that could be placed in 
the cells currently containing {4, 8} (since the cells aren’t on adjacent diagonals) nor 
do {3, 9} (where the smaller car on the lower diagonal is not northeast of the larger 
car.) The dinv of the parking function in Fig. 2 is 14, including the pairs: {3, 4}, {1, 2}, 
{1, 9}, {2, 9}, {1, 6}, {2, 6},{6, 9},{4, O},{1, 5},{2, 5}, (5, 9}, (4, 5}, (5, 6}, and {6, 8}. 
We’ ve bolded the southwest car in each pair, for reasons which will be more apparent 
later. 


If two cells are positioned so that they could create a diagonal inversion if the two 
numbers contained within them are in the “correct” order (as, for example, in Fig. 2 
both the pair {3, 4}, which makes an diagonal inversion and {1, 4}, which does not), 
then the cells are called attacking. 


Definition (word). We read the word of a parking function by reading along diago- 
nals from highest to lowest, within a diagonal reading from right to left. 
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Fig. 3. These three parking 

functions have weights FA iz eq id 4 
tFi1),F ig}, and qk, 

ee A Bo Ee 


In particular, note that the word of a parking function is a permutation in G,, and 
thus has a well-defined inverse. 


Definition (ides). The i-descent set of a permutation is defined to be the descent 
set of the inverse of a permutation, that is, the set of i in a permutation such that 
i + | occurs before 7. The i-descent set of a parking function is the descent set of the 
inverse of the word of a parking function. 


Example 3. The word of the parking function in Fig. 1 is [1, 2, 3, 4,5, 6, 7, 8, 9]. 
It’s 1-descent set is {1, 3, 4, 5, 7}. 


Example 4. There are three parking functions of size 2, pictured in Fig. 3. Comput- 
ing their weights, we get that 


DH2[X; gq, t] =tFay + Fo t+ Fy = (4+ 9)sa.y +52). 


Thus by the Shuffle Theorem, DH) contains a submodule of degree 0 that is isomor- 
phic to the trivial representation of Gz and two submodules, one of degree | in the 
x variables and one of degree | in the y variables, that are isomorphic to the signed 
representation. 


While these statistics (particularly dinv) may not seem particularly natural, their 
discovery was important not just to the Shuffle Theorem, but also to the combi- 
natorics of the Macdonald polynomials. As in the Shuffle Theorem, combinatorial 
interpretations of Macdonald Polynomials require two statistics, one of which (maj) 
is relatively natural, and another (inv) whose discovery was motivated by dinv in 
parking functions. Haglund [27] describes the process by which he first defined inv, 
which is a more complicated statistic than dinv, from dinv on parking functions. 


2.1 Shuffles and the q, t Catalan 
For convenience, we name the sum in the right-hand side of (1). Let 


TI, (X; q; t) = SS parea(m) gdinv(r) Fr (PF): 
TEPF, 


TI, (X; q, t) is a positive sum of LLT polynomials, which are Schur positive poly- 
nomials developed in Lascoux et al. [39] and Leclerc and Thibon [40]. While the 
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original definition of LLT polynomials (in terms of a statistic on ribbon tableaux) 
does not generally “look” very similar to parking functions, work in Schilling et al. 
[52], simplified independently by Haiman and Michelle Bylund, gives an equivalent 
definition which is used in Haglund et al. [30] to show that IT, (X; g, t) is Schur pos- 
itive. Since in particular IT,,(X; q, t) (as a sum of LLT polynomials) is symmetric, 
it makes sense to consider the Hall inner product of IT, (X; qg, t) with hy, --- hx,. Let 


Sh(ky.-++ ks) = {lye ey} fey Hye ky Hp LU ky Hee kg yp Hh ky thy Hee Hs} 


give the set of shuffles (words with 1, --- , k; occurring relatively in increasing order, 
as are {kj + 1,--- , ky + ky} and so on). Then basic symmetric function theory gives: 


(Wn (Xx; q; t), hk, ne hy,) = x ua gael 


TEPF, 
word(7)€Sh(ky,-+ ,ks) 


and motivates the name of the Shuffle Theorem. Similarly, 


(Ty (x: qs t), en) = > gain) Sn 


nTePF,, 
word(7)=(n,n—1,--- ,1) 


If we restrict ourselves to this case, since the word is completely determined, we 
can just leave the car numbers off entirely from the diagram; the result for each such 
parking function is a unique Dyck path. Note that this implicitly defines the dinv 
and area of a Dyck path, with the area being the sum of the cells below the path as 
before and the dinv simplifying, since the conditions on the relative size of the cars 
are always fulfilled, to the number of attacking cells. Since Dyck paths are counted 
by Catalan numbers, this motivates the name used to describe this case: the q, t- 
Catalan Theorem, which was first proved by Garsia and Haglund [16]. Historically, 
dinv (and before that bounce, as described below) was developed first on Dyck paths, 
not on parking functions. Since we describe the Shuffle Theorem here first, and then 
give the q, t- Catalan case as a specialization, our presentation here is in the reverse 
order of the historical development, where the Catalan case was generalized to the 
Shuffle Theorem. 


3 Alternate Formulations of the Shuffle Conjecture: The ¢ 
Map and the T Map 


One map that is particularly important to the proof of the Shuffle Theorem is the ¢ 
map, which shows II,,(X; qg, t) can equivalently be computed as a sum over “diag- 
onally labeled Dyck paths”—a variant on parking functions. A second map, the [° 
map, which when restricted to Dyck paths agrees with the ¢ map, proves a particular 
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symmetry of the parking functions and leads to a natural generalization of the shuffle 
conjecture. 


3.1 The ¢ Map on Dyck Paths 


There are several equivalent formulations of the ¢ map on Dyck paths. Perhaps the 
simplest (although not the original) definition comes from the language of sweep 
maps in Armstrong et al. [3]. Here, as elsewhere, we find it convenient to picture 
a Dyck path as not in ann x n grid, but ann x (n+ 1) grid, with the Dyck path 
remaining above the diagonal from the bottom left corner to the top right (so that the 
diagonal is tilted just slightly clockwise to “break ties”). There are the same Catalan 
number of such paths in either grid, with “Dyck paths” in then x (n + 1) grid being 
created from ones in the standard square grid by adding a final east step. 

We will need an ordering on each of the individual north and east steps in a Dyck 
path. Such an ordering will be defined by studying the south end of a north step and 
the west end of an east step; that is, our ordering on the steps is based on the position 
of the first point we would touch each step if we were to trace along the path from 
bottom left to top right. 

Draw 7 as a Dyck path in n x (n + 1) grid. Then create a new path C(7) by 
sweeping the main diagonal upward, and recording the new north and east steps as 
a new path as they (or rather their south and west endpoints) are encountered. 

While this description does not make it “obvious,” ¢ is a bijection from the Dyck 
paths of size n to itself. Moreover, it has a predictable effect on the statistics of the 
Dyck paths; in particular if 7 has area a and dinv b, ¢(7) has area b and bounce a, 
where bounce is a third statistic on Dyck paths (viewed on the n x n grid.) 


Definition. To compute the bounce of a Dyck path 7, trace a new Dyck path, called 
the “bounce path,” starting at the point (0, 0) and moving upward: 


(1) Turn right when you hit the west endpoint of any east step of 7. 
(2) Resume an upward path after intersecting the line x = y. 


Repeat these two steps until the path goes from (0, 0) to (n, 7). 
Finally, if the bounce path intersects the line x = y at the points 


(0, 0), (n a ij,n = it), aan: | (n a ig, n — Ik), (n,n), 


then by definition 
bounce(7) := iy) +--+ + ig. 


Example 5. In Fig.4, the Dyck path on the right has bounce 1 + 6 + 8 = 15 


Using this definition and D, for the set of Dyck paths of size n, as related in 
Haglund and Loehr [29, pg.9], we thus have 
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Fig. 4 7 and ¢(z). The diagonal line is slightly tilted in the preimage as it occurs in the definition 
of the sweep map and emphasized the extra east step, but kept the diagonal line x = y in the image, 
as it occurs in the included description of ¢~!. The dashed line gives the bounce path. The numbers 
along the right correspond to after how many bounces the given row was crossed by the bounce 
path, but also indicate the diagonal containing the preimage of that north steps in the Dyck path on 
the left 


Theorem 4 (Garsia, Haglund, Haiman). 


= peta dive) = > pbouncelm gareat). (2) 


TED, TED, 


A nice detail of the bijection is that area under the bounce path of ¢ (7) corresponds 
to primary diagonal inversions in 7 and area under ¢(7) but above the bounce path 
corresponds to secondary inversions. Extending this observation slightly gives the 
inverse map, most easily stated as onto the set of parking functions with word [n, n — 


1,--- , 1]. dn the image, ¢(7), it’s useful to consider adding the number from bottom 
to top to get a parking function. In 7 = ¢~'(C(7)), we’ll find the result has word, 
as read along the diagonals, [n,n — 1,--- , 1].) In particular, if the ith north step 


(number from the bottom) in ¢(7) was in a row crossed after the bounce path’s 
jth return to the line x = y (noting that the first step or more will thus be assigned 
the number 0, since the bounce path has not yet “returned’’), then i was in the jth 
diagonal of 7. (As a quick exercise to check for understanding, the interested reader 
may wish to check that the sum of such j must sum to the bounce of ¢(7), so that 
bounce(¢(7)) = area(7), as the sum of the diagonals containing each car.) Since 7 
originally corresponded to a Dyck path, the cars in any diagonal should be placed 
in increasing order from southwest to northeast. The resulting diagonal inversions 
in 7 (the primary diagonal inversions) account for area under the bounce path. We 
still need to account for the relative order of cars (and in particular which occurs 
in which row) in distinct diagonals. In particular, this is uniquely determined by the 
prescription that in 7, cari is southwest of 7 smaller cars in the next lower (southeast) 
diagonal if in ¢(7), there are j cells in the ith row that were above the bounce path. 
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Fig. 5 7 and ¢(7). Note that for any corner in ¢(77) like the one marked with a «, the car below the 
corner is smaller than the car to the right of the corner 


Example 6. The ‘*’ in 7 and the ‘@’ in ¢(7) in Fig. 4 correspond to the same cell. 
(Note that the ‘x’ is in the first cell that is read in the reading word of 7 and thus 
would be labeled n = 9, while the ‘@’ occurs in the 9th row from the bottom.) The 
single north step in 7 northeast of ‘x’ is exactly one diagonal lower, so it corresponds 
to a secondary inversion. It thus corresponds to a single cell above the bounce path 
in ¢(7) in the row containing ‘@’, in particular the cell containing ‘@’ itself. 


3.2. ¢ on the Parking Functions 


There are two well-known alternate characterizations of T,[X; q¢,t], both which 
extend ¢. We start with the summation tied to the bijection (again) called ¢ in the 
literature. The image of ¢ on parking functions is not a parking function. The image 
does not seem to have a standard name in the literature; we’ll refer to it here as the 
diagonally labeled Dyck paths (DLD,,). The diagonally labeled Dyck paths are also 
represented in a lattice diagram, which again contains a Dyck path. Each Dyck path 
contains a permutation of 1; this time the numbers (which we will here still call cars) 
are placed along the main diagonal. Rather than an increasing column condition, the 
only restriction on the values of the cars in DLD,, is based on certain corners in the 
Dyck path. In particular, if the underlying path contains an east step followed by a 
north step, the car below the east step must be smaller than the car to the right of the 
north step. 


Example 7. See the right side of Fig. 5 for an example. 


Definition. For 7 € PF,,, we define ¢(7) as: 


e First, if D is the Dyck path in 7, create ¢(D). 
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e Add cars to the diagonal of ¢(D) from northeast to southwest, labeling the cells 
with the reading word of 7. 


The result is ¢(7). 
Example 8. See Fig.5 for an example. 


The bijectivity of ¢ on parking functions follows simply from the bijectivity of 
¢ on the Dyck paths. Moreover, we can again define statistics in the image which 
correspond to area and dinv in the preimage. Since area is constant on Dyck paths 
and we have defined ¢ on Dyck paths to send area to bounce, it is clear that one such 
statistics should be the bounce of the path in ¢(7). The dinv of a parking function 
is mapped under ¢ to the area of the result only when every attacking pair of cars 
in 7 creates a diagonal inversion, i.e., when the parking function has reading word 
n,n —1,---, | and thus corresponds to a Dyck path. We need a new statistic, area’ 
which is less than or equal to area. 


Definition. The area’ of 7 € DLD,, is the number of cells c between the Dyck path 
and the main diagonal such that the car below c is smaller than the car to the right 
of c. 


Example 9. In the diagonally labeled path on the right of Fig. 5, 
({3, 4}, {4, 5}, {4, O}, (1, 2}, (1, 5}, (1, 6}, (1, 9}, (2, 5}, {2, 6}, (2, 9}, {5, 6}, {5, 9}, (6, 9}, (6, 8}) 


all contribute to the area’ of 14. Note that these pairs are exactly the pairs that create 
dinv in the parking function on the left. The underlying Dyck path on the right has 
bounce 5, which corresponds to the area of 5 on the left. 


Finally, we have the theorem: 
Theorem 5 (Haglund and Loehr [29]). 


+ phen tunin) = > phomnectay gates (3) 


TEPF,, TEDLD,, 


3.3. Ton the Parking Functions 


Unlike ¢, P is a map from PF’, to itself. This generalization leads naturally to rational 
parking functions and suggested a parallel story to the diagonal harmonics in other 
types. We’ll describe it two ways, once as a combinatorial generalization of ¢ on 
Dyck paths as described above and once as a composition of two bijections to a set 
of intermediate objects (a particular subset of affine permutations). 
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3.3.1 A Direct Combinatorial Description 


Since all the attacking cells in a Dyck path create diagonal inversions but only some 
of them create diagonal inversions in a parking function, it’s reasonable to suspect 
there might be a generalization of bounce that generally lowers the statistic depending 
on the location of cars and is equidistributed with area of the parking functions. In 
fact, such a statistic exists and is referred to in the literature as pmaj. While it can 
no longer be pictured using the geometry of a bouncing ball, when 7 € PF’, has car 
labels increasing from the bottom row to the top row, pmaj(z) is the same as the 
bounce of the underlying Dyck path. (As remarked previously, a close look at the 
inverse map described in Sect. 3.1 shows that we were implicitly injecting the Dyck 
paths into the parking functions in two ways: in the preimage of ¢, we considered them 
as parking functions with word [n,n — 1,--- , 1] but in the image, we considered 
parking functions with cars increasing from the bottom row to the top.) One way we 
might have equivalently described bounce is by a recursive procedure, starting by 
assigning any north step/car in the first column bounce 0. We then assigned bounce 
to cars in increasingly higher amounts. After assigning bounce /, by returning to the 
line x = y, we count the number of cars already assigned bounce i or less. If that 
number is 7, we then assigned bounce i + | to cars which are: 


e inthe j + 1th column or less and 
e have not yet been assigned bounce. 


We then repeat the process, this time assigning bounce i + 2. Pmaj can be assigned 
similarly, but we should expect that some cars will be assigned a pmaj earlier in the 
procedure (and thus a lower pmaj) on the basis of their value. 

Formally, for a parking function 7, the procedure for assigning pmaj is: 


Assign every car in the first column a pmaj of 0. Set p (which we’ll use to keep 
track of the current pmaj we assign) and k (the number of cars assigned a pmaj of 
at least p — 1) to 0. 

For every car c not yet assigned pmaj, count the number of larger cars already 
assigned pmaj exactly p, call it 7. Ifc is in column k + j + | or less, assign c the 
pmaj p. (It’s best to work with the cars c in decreasing order of size, although one 
can equivalently repeat this step until no new cars are able to be added.) 

Increase p to p + 1, and increase k to include the number of cars added in the 
previous step. 

Repeat (2) and (3) until all the cars are assigned a pmaj. 


We will use pmaj(c, 7) for the pmaj of the car c in the parking function 7. 


Definition (pmaj). The pmaj of a parking function 7 (pmaj(z)) is the sum of the 
pmaj assigned to each car. 


Example 10. Consider the parking function on the right of Fig. 6. Car 7 is imme- 
diately assigned a pmaj of 0 since it’s in the first column. Cars 9 and 8 are larger 
than 7 and so are not assigned pmaj (because they aren’t in the (0 + 0+ 1)st or first 
column). Similarly, 6 and 5 aren’t in the (0 + 1 + 1)th or second column. However, 
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car}/l 2 3 4 5 6 7 8 9} car 
area}O 0 0 0 1 1 0 2 1) pmaj 
dinv}1 010 5 4 0 1 2) area 


Fig. 6 7 (repeated here from Fig.2) and (7) 


4 and 3 are both in the 2nd column (and less than 7), so they are both assigned a pmaj 
of 0. Next, 3, 4, 7 (which have already been assigned pmaj this round) are all greater 
than | and 2, which are in the 4th column, so since0 + 3+ 1 > 4and0+4+41>4, 
2 and 1, respectively, are assigned pmaj 0. This ends the cars with pmaj 0. 

Next, we’ve assigned five cars pmaj 0, so any unassigned car in the 6th column or 
less has pmaj |, including cars 5, 6, and 9. Moreover, once car 9 is assigned a pmaj 
of 1, we compare the sum of the five previously assigned cars and the one larger car 
than 8 (the 9) which was recently assigned a pmaj of 1. Since 5+ 1+ 1 < 8, we 
cannot assign 8 a pmaj of 1. 

In a final assignment, we may use the eight cars assigned pmaj one or zero to 
justify a pmaj of 2 for car 8. 


The observant reader may notice that the diagonal containing c in the parking 
function on the left of Fig.6 is in every case the pmaj of c in (7). This is not a 
coincidence, but rather the starting place of the map we’ll use to define I’. We’ve 
implicitly assigned pmaj to each car; we’ll do the same for area (in two ways, once 
for the preimages and once for the images) and dinv. The resulting assignments will 
be equidistributed and used to define the bijection. 

For 7 in the preimage of I’, first assign the area of car c (area(c, 7)) as the number 
of complete squares weakly to the right of c and left of the line x = y (ie., the 
diagonal of the car c). Next, we must decide which of the two cars in a dinv pair 
should be assigned the diagonal inversion. We will find it useful to assign the dinv 
of a car c (dinv(c, 7)) as the number of diagonal inversion pairs containing c and in 
which c is the further southwest of the pair. In particular, this means that a primary 
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diagonal inversion is assigned to the smaller of two cars, but a secondary inversion 
is assigned to the larger car. 


Example 11. In Example 2, we’ve bolded the cars which are being assigned dinv in 
each pair of diagonal inversions in the left of Fig. 6. 


For (7), we’ve already described the assignment of pmaj. Although the area of 
the parking function is defined the same way on each side, we need to assign area to 
individual cars differently in the image. In particular, the area of a particular car is 
closely tied to our assignment of pmaj and how many columns right c can move and 
still have the same pmaj. In particular, here we assign 


areap(c, I'(7)) 
= #{d| pmaj(d, "(m)) < pmaj(c, T(7)) + #{d > c| pmaj(d, [(7)) = pmaj(c, T(7))} 
—column(c, '(7)) + 1. 


where column(c, I"(7)) gives the column containing c (as usual numbering from left 
to right). Then 


area(I'(7)) = area,(c, (7)), 


c=l 


since if we sum over cars first sorting by their pmaj (small to large) then by 
car (large to small), we will find #{d| pmaj(d, (7)) < pmaj(c, [(7))} + #{d > 
c| pmaj(d, '(7)) = pmaj(c, '(77))} gives us the sequence 0, 1, 2,--- ,2 — 1 which 
sums to the number of cells strictly above the main diagonal in the lattice. Meanwhile 
—column(c, I'(7)) + 1, summed over all cars c, is the number of cells above the 
Dyck path (usually called the coarea), so the sum of all terms is the area between the 
path and the diagonal as claimed. 

Recursive procedures detailed in Haglund and Loehr [29] and Loehr [42] show, 
using slightly different language, that the assignment of the diagonal and dinv of every 
car (area(c, 7) and dinv(c, 7), respectively, for every car c) in this way uniquely 
defines a parking function. In particular, if for a particular parking function 7, 
de = area(c, 7) and b, = dinv(c, 7), then only 7 has area sequence (a1, d2,--- , Gn) 
and dinv sequence (b;, bz, --- , b,). Similarly, they give a recursion on parking func- 
tions using the area and pmaj statistic which shows that a parking function 7’ is 
uniquely defined by the sequences (pmaj(c, 7’))"_, and (areap(c, 7’))?_,. More- 
over, they show that if 7 had area sequence (a), d2,--- ,da,) and dinv sequence 
(bi, bo,--- , b,), then there exists (uniquely by the previous comment) a parking 
function 7’ with pmaj sequence (a1, a2, +--+ , d,) andareap sequence (bj, bz, --- , bn). 
We define ['(7) = x’. Thus we have the theorem: 


Theorem 6 (Loehr [42]). 


2 pre aw) = > gpmaln) eeetn) (4) 


TEPF, TEPF,, 
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Since the left-hand side of (4) is exactly (II, (X; q, ft), e7), we have the following 
corollary: 


Corollary 1. 
(Tn (X3 q, 1), ef) = (Mn(X3 1g), 4). 


The symmetry of the x;,--- , x, and y,,--- , y, makes it obvious that the Shuffle 
Theorem implies I], (X; g, t) = 1, (X; ft, g), but the corollary above is the closest 
any direct combinatorial proof has come to proving the symmetry. 


3.3.2 Affine Permutations, Hyperplanes, and an Alternate Description 
of T 


While the above description of I was historically identified first and gives a great deal 
of information about the three historically studied statistics on the parking functions, 
there is another description of I" as the composition of two bijections. These bijections 
were originally defined as bijections from parking functions to regions of the Shi 
hyperplane arrangement, but can be more explicitly defined in terms of a subset of 
affine permutations which are in bijection with these regions. Much of this work is 
first stated in Armstrong [1] and Armstrong and Rhoades [2]. We base this section 
on the more general bijections in Gorsky et al. [22], adding a few minor changes 
documented below, and will return to the more general bijections later. 


Definition (affine symmetric group). Recall that the affine symmetric group G,, can 
be identified with bijections w : Z — Z subject to the restrictions: 


(1) For all integers x, w(x) = w(n+ x). 
(2) Dili w@ = SP. 


Multiplication in the group is then composition of functions. 


The affine permutations are generally represented using window notation 
[w(1),--- , w()]. Affine permutations are generated by sg, --- , 5,_, where: 


so = [0,2,3,4,--- ,n-—I,n4+ 1] 
sj =[2,1,3,4,--- ,n-—1,n] 
s2 = [1, 3, 2,4,--- n—I1,n] 


Sn-1 = [0, 1, 2,3,4,--- ,n,n— 1). 
G,, can be equivalently defined by relations on these generators: 


(1) ee =), 
(2) sjs; = sjs; fori— j A+1 mod n. 
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(3) sjsjs; = sjsjs; fori — j =+1 mod n. 


Definition (m-restricted). Call w € G,, m-restricted if foralli < j,w)-wpF 
m. Denote the set of m restricted affine permutations "G,,. 


Then to define the bijection A: PF nom mtH1G (similar to a map of Athanasiadis 
and Linusson [5] but here given in the language of Gorsky et al. [22]), we start with 
an assignment of “rank” to each of the cars. (In fact, it’s easiest to assign a rank to 
each square in the lattice diagram, although we only care about the rank of the cells 
containing cars.) We assign rank according to the following convention: 


(1) The bottom left cell has rank 0. 
(2) Every step northward corresponds to an increase in rank of n + 1. 
(3) Every eastward step corresponds to a decrease in rank of n. 


(Note that the choice of rank of the bottom left cell is arbitrary, and there are different 
conventions depending on the author.) 


Example 12. By this convention, looking at the cars in the parking function on the 
left of Fig. 6, car 7 has a rank of 0. The empty cell above 7 has a rank of 0 + 10 = 10, 
so the cell containing car 3 has a rank of 10 — 9 = 1. The following chart gives the 
ranks for all the cars in the parking function: 


12345678 9 
78151113021 15 


car 
rank 


To create A(x): 


e Create a list of the ranks of each car in 7 (ordered by the cars): 
w = [rank(1), rank(2),--- , rank(7)]. 


e was created will generally not be an affine permutation since }~"_, w(i) 4 mee) 


Rescale w (adding 4 (24 _ ys wi )) to each term in the window). 


The result of the rescaling is A(n). 


Example 13. 7+8+1+5+4+114+13+0+421+4 15 = 81 isthe sum of the ranks 
of the cars in the parking function on the left of Example 6. Since our final permutation 
w should sum to 45, we need to subtract 36/9 = 4 from each rank to get the final 
permutation. Thus A(r) = [3, 4, —3, 1,7, 9, —4, 17, 11]. 


Remark. For this algorithm to work, it is necessary that the rescaling factor to 
be an integer. To see this is indeed the case, note that every cell in a row of a 
parking function has the same rank modulo n, no two rows contain ranks of the same 
modulus, and finally that each car is in a different row of the parking function. All 
of this is guaranteed by the fact that n and n + 1 are coprime. Why do we only get 
n+ 1 restricted affine permutations? Cars with ranks that are exactly n + 1 apart 
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correspond to cars that are in the same column, and thus if wii) — w(j) =n + 1, 
car i must be in the cell directly above car j and thus (by the column increasing 
condition) must correspond to a larger car. 


Next, we define a second (much simpler) bijection PS "+! 6, & PF, where 
the result is a parking function in one-line notation (or equivalently the ith term gives 
the column of the ith car). In particular, for 1 < i <n, we can define the 


PS(w); = L+H j > 10 < w@) —w(j) <n4+ 4h. 


It is important for this definition that we keep track of bounded inversions involving 
indices j that are outside the window. 


Example 14. Continuing the previous example, notice that 
PS 0 A(n) = PS([3, 4, —3, 1, 7,9, —4, 17, 11) 


is the parking function (4, 4, 2, 2, 3,3, 1, 8,4), which corresponds exactly to the 
parking function on the right-hand side of Fig.6. For example, the 5th term in the 
resulting parking function is 3. This corresponds to the inversions between the 5th 
term in w and the 12th and 16th. Note that this does not include an inversion between 
the 5th and the 7th, since 7 — (—4) = 11 > 9+ 1. 


Our map A is closely related to the map A in Gorsky et al. [22]. Letting inv 
gives the inverse map on affine permutations, we’ve given a combinatorially simpler 
version (from this perspective at least) by defining A= invoA"!, Similarly our 
following map PS = PS o inv in the language of the same paper, a map which 
they briefly refer to as SP. Gorsky et al. [22] state that PS o A™! is equal to ¢ 
when restricted to the Dyck paths (a statement which is essentially due in this case 
to Armstrong [1]). It’s not hard to see that PS o A7! = PS o A nor that if we look 
at the resulting map on all parking functions, the result is actually T. 

Both A and PS (at least in this classical case) can be closely identified with 
classical maps from alcoves in a certain hyperplane arrangement, called the Shi 
arrangement, to the parking functions. 


Definition. Let Hi ; gives the hyperplane defined by x; — x; = k restricted to 
V = {x € R"\x, +--- +x, =O}. 
Then, the affine braid arrangement is the set of hyperplanes 
B, ={Hf,:0<i<j<n,keZ}. 
The Shi arrangement is the set of hyperplanes 


S, ={Hf,:0<i <j <n,ke {0, 1}). 
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Fig. 7 Shi arrangement, with alcoves labeled by 4 restricted affine permutations of 3 


Alcoves (connected regions bounded by the hyperplanes) in B, are in bijection 
with affine permutations of size n; alcoves in S,, are in bijection with only those 
permutations which are n + | restricted. The “fundamental alcove” satisfying 


Xp > XS > XH > xX 1 


corresponds to the identity permutation [1,2,3,--- ,]. From there, new alcoves 
are labeled by reflection across H is corresponding to multiplication on the left 
by so, and He 4 corresponding to multiplication on the left by s;. (Some sources, 
including Gorsky et al. [22], but not Armstrong [1] biject S,, to the n + 1 bounded 
affine permutations, which are the inverse of the n + 1 restricted permutations. In 
this context, one must label the regions by the same method, except that reflections 
across hyperplanes correspond to multiplication on the right, not the left.) If we 
then compose this labeling with A, we get one labeling of the alcoves of the Shi 
arrangement by parking functions. This labeling is quite similar (but not identical 
to) the labeling of Athanasiadis and Linusson [5] of the Shi arrangement by parking 


functions. Similarly, we can label the regions by composing our labeling with PS . : 
The result (in this case) is better known as the Pak-Stanley labeling Stanley [56]. The 
region where x; > --- > x, is labeled by increasing partitions corresponds under the 
Pak-Stanley labeling to Dyck paths. 


Example 15. See Fig.7 for a concrete example of the labeling. 
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4 Proof of the Shuffle Theorem 


As is perhaps not surprising for a theorem two decades in the making, many 
researchers have played important roles in its proof, and we highlight a few of these 
results, including those results particularly important to the ultimate proof of the 
Shuffle Theorem. 

A first key result, leading to the Shuffle Theorem, is the following result in Haiman 
[33]: 


Theorem 7. Let DH,,[X; q,t] give the bigraded Frobenius characteristic of the 
diagonal harmonics. Then 
DH, [X; q,t] = Ven. 


The operator V here is an eigenoperator of the Macdonald H LX; q, t] first defined 
by Francois Bergeron. (See Bergeron et al. [9] for an extended discussion of the 
operator.) If py’ is the conjugate partition of j1, and 


k 
n(u) = >) wii — 1), 
i=1 


then V is defined by 


VALX; ¢, t] = "gq" AX; ¢, t). 


Haiman’s proof of Theorem7 is involved, requiring background both in algebraic 
geometry (relating to the Hilbert scheme of points in the plane) and algebraic com- 
binatorics; parts of the proof were suggested earlier by Procesi and the formula was 
first conjectured in Garsia and Haiman [14]. 

Once Theorem 7 had been established, attention turned to establishing that 


Ven = TT, LX; ¢, ¢], (5) 


with the ultimate goal of showing, using Theorem 7, that the Frobenius characteristic 
of the diagonal harmonics was a weighted sum of parking functions. Progress toward 
the proof began to follow the same general steps: 


(1) On the symmetric function side, several refinements of e, were identified, includ- 
ing e, = )°, Ey, in Garsia and Haglund [15], and e, = aes C, in Haglund 
et al. [24], which after application of V, appeared to have nice Schur function 
expansions. 

(2) Simultaneously, on the combinatorial side, a subset of the parking functions were 
identified which corresponded to the symmetric function refinement. 

(3) Finally, an inner product with a product of elementary and complete homoge- 
neous symmetric functions was taken with both the symmetric function and 
combinatorial refinement and the resulting polynomials were shown to be equal. 
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On the combinatorial side, such an inner product just restricts the parking func- 
tions to those whose words were a certain type of shuffle. 


Results of this sort were offered as evidence for the Shuffle Theorem, since they 
implied that some coefficients in the expansion of Ve, and T,[X; q, t] in various 
symmetric function bases were the same, and the ultimate goal was to complete the 
process for an arbitrary length product of h’s (or e’s), thereby proving (5). 

Besides the previously mention proof of the g, t Catalan, the first paper of the 
type was Haglund [26], who showed that 


(Ven, €ahb) = > garea(n) 9 dinv(™) Fe esp), ca] 
TEPF, 


by refining the conjecture to identify those parking functions with k cars on the main 
diagonal. 
The results were improved with a conjecture of Haglund et al. [24]: 


VCa = > eT a Fiaes(P F): (6) 


TEPF, 
touch(m™)=a 


Here Cy is a symmetric function created by the modified Hall—Littlewood vertex 
operators and touch(7) is the “touch composition” of a parking function. 


Definition (touch composition). In particular, split the cars of 7 into an ordered set 
partition, adding cars by row and starting a new set every time the Dyck path returns 
to the main diagonal. The size of the sets gives a composition of n. 


Example 16. In Fig.2, the composition of the parking function on the left is 
[1, 4,2, 1, 1], counting the sets ({7}, {3, 5, 8, 6}, {4, 9}, {1}, {2}). 


Using (6), recursions were found in Hicks [35], Garsia et al. [18], and Garsia 
et al. [19] which demonstrated the conjecture was true after taking various inner 
products, but there seemed to be no hope of extending this method of attack to prove 
the conjecture, without further refinement. 

Recently, Carlsson and Mellit [12] announced a proof of the Shuffle Theorem. 
Their 35 page proof was expounded on in a 60 page, nearly self contained, series of 
lecture notes Haglund and Xin [25]. As the length of these both suggest, the proof 
is quite involved. Moreover, while the mathematics is a combination of symmetric 
function manipulation and bijective combinatorics, many of the details are quite 
technical. To the interested reader with background in algebraic combinatorics, we 
suggest Haglund and Xin [25] as a worthy resource. The proof requires a number of 
plethystic manipulations; beyond the lecture notes, the uninitiated reader may wish 
to consult Loehr and Remmel [43] for a better understanding of the useful notation. 
We make the following observations about the proof for the reader interested in a 
few of the details. 
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Carlsson and Mellit’s proof picks up where Haiman’s proof left off, in that they 
show: 


Theorem 8 (Carlsson and Mellit [12]). 
Ven = 1, LX; q; t]. 


In fact, their proof refines and expands the conjecture of Haglund, Morse, and 
Zabrocki stated in (6) above. Carlsson and Mellit do not work directly with the 
right-hand side of (6) but rather with its image under the ¢ map. Additionally, they 
use the opposite ordering of cars everywhere, so that columns are decreasing from 
bottom to top, the relative car size conditions on dinv are reversed as are the corner- 
based restrictions in DLD,, etc. Since they work after applying the ¢ map, they first 
must find an analogue of touch composition in the image of ¢; their definition of 
touch' is (by necessity) considerably more complicated than touch. They call the 
image of the right-hand side of (6) under ¢, D,(q, t), and their proof directly shows 
then that D,(q,t) = VCa. 

As with previous progress on the Shuffle Conjecture, a key idea in the proof 
is that further refining the conjecture makes it easier to prove. Carlsson and Mellit 
specifically identify symmetric function operators which give the weighted sum of all 
parking functions with a given Dyck path, further identifying even partial Dyck paths 
in some well-defined sense. In particular, they identify the set of DLD,, with a given 
path, and define “raising and lowering” operators (d, and d_), which correspond in 
the symmetric functions to adding an east or a north step to a partial path. Expressing 
D.(q, t) more cleanly in terms of these operators applied to a recursively defined 
polynomial (i.e., rather than as a sum of Dyck paths) leads to an expression that they 
show can be rewritten as VC,,; the mechanics of showing this equivalence is quite 
involved, involving a careful analysis of a number of operators and relations, and is 
not “just” a string of equalities. 


5 Extensions of the Shuffle Theorem 


Several natural extensions of the Shuffle Theorem are areas of active research. In 
this final section, we’ll briefly describe several of these and give references for more 
complete details. 


5.1 Rational Shuffle Theorem 


One of the better known and more natural extensions of the Shuffle Theorem is 
the Rational Shuffle Theorem. From the perspective of algebraic combinatorics, the 
conjecture can be motivated from a natural extension of the [ map as given in terms 
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of affine permutations and hyperplane arrangements. Gorsky et al. [22] show that 
PS and A can naturally be extended to the set of not just n + 1 restricted affine 
permutations, but the set of m restricted permutations when m and 7 are relatively 
coprime. The resulting parking function image or preimage naturally becomes the 
set of so called rational parking functions, PF’,,;,, which can similarly be pictured 
as occurring in m x n lattices containing paths of north and east steps which remain 
above the line y = n/mx. Again, one can determine the area of 7 € PF yy/, as the 
number of squares below the path and strictly above y = (n/m)x. Moreover, dinv(7) 
can be defined as area(I'(71)). A slight complication in this definition is that PS, when 
it was defined, was not known to be bijective for all m, n, but recent work of Thomas 
and Williams [57] suggests this is at least the case when 7 is a parking function 
with reading word [n,n —1,--- , 1]. Moreover an equivalent (if far less elegant) 
definition for dinv can be worked out directly on 7, as was first defined in Hikita 
[36] and was simplified in Hicks and Leven [34]. Together this suggests one could 
consider a combinatorial sum of weighted rational parking functions: 


di 
Thinn LX; q; t] = > pansy mn Fides(r) 5 
TEPF n/n 


with some natural statistics generalizing those in the classical case. A reasonable 
question is whether one should study the sum (i.e., whether the sum is of independent 
interest). By expanding our attention from the Shuffle Theorem to this more general 
polynomial, a number of interesting additional connections become apparent. Hikita 
first investigated IT,,,,[X; qg, t] as the Frobenius image of a module arising naturally 
from the study of affine Springer fibers. Moreover, in Gorsky and Negut [20] a 
conjectured symmetric function is identified, leading to a natural analogue of the 
Shuffle Theorem: 


Theorem 9 (Rational Shuffle Theorem). 


Omn(—1)” = Tn LX; q; t]. 


A convincing reason to investigate Q,,,,(—1)” is its conjectured connections to irre- 
ducible modules in the rational Cherednik algebra as explained in Gorsky et al. [21] 
and its proven connections (see both Gorsky et al. [21] and Oblomkov et al. [46]) to 
knot invariants of the torus, where Q,,,,(—1)” gives a concrete way of computing 
the “superpolynomial knot invariant” (which generalizes the well-known HOMFLY 
polynomial) for torus knots. For the combinatorial audience, we suggest Haglund 
[23] for an overview of these connections, Bergeron et al. [10] and Bergeron et al. 
[11] for a clear statement of the conjecture, an explicit definition of Q,,,(—1)”, and 
what is known when m and n are not relatively coprime, and finally Mellit [45], for 
his recently released proof of the conjecture. 
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5.2 Additional Conjectures and Theorems Related to V 


The Macdonald eigenoperator V first appeared in the context of the Shuffle Theorem, 
applied naturally to e,. Since then, experiments have shown that V applied to many 
natural symmetric function basis elements is Schur positive; the result has been 
additional nice combinatorial formulas, currently either proven or conjectured. A 
nice summary of most of these conjectures occurs in Loehr and Warrington [41], 
although several nice works inspired by recent developments on the Shuffle Theorem 
occurred after its publication, including for example Sergel [53] and Wilson [58]. 

Additionally, it seems that V itself has an interesting generalization, as first defined 
in Bergeron et al. [9]. Recall that V is defined by 


VA,LX; ¢, t] = "gq" AX; , t). 


Next, if jz is a partition of k, "gq" is equal to e,[B,,], where B,, is a weighted 
sum of cells in the Ferrer’s diagram of ju and [-] indicates plethystic substitution. 
Then several works have examined a new family of eigenoperators of the modified 
Macdonald polynomials also defined in Bergeron et al. [9] for a general symmetric 
function f by: 7 7 

Af A,X; ¢,t] = fLBvALX: @, t]. 


It follows that V f = A,., f if f is a homogeneous symmetric function of degree k. 
Interestingly, for various natural choices of f, A fe, appears to still be Schur positive, 
with Haglund et al. [31] giving one of the best-known conjectures in this area for a 
combinatorial interpretation of the polynomial and Rhoades [49] and Romero [50] 
providing evidence for it. In some cases (such as Rhoades and Wilson [48]), there are 
conjectured modules whose bigraded Frobenius characteristics are believed to give 
a representation theoretic interpretation for the symmetric function as well. There is 
not yet one known combinatorial perspective for studying either V f or Aye, as f 
varies over the symmetric function bases. 

Several additional generalizations of the diagonal harmonics can be found in 
Bergeron [6], which examines generalizing the G,, action to a finite complex reflec- 
tion group and working with an arbitrary number of sets of variables to define a new 
family of spaces generalizing the diagonal harmonics. The work shows the resulting 
spaces have Frobenius characteristics with some nice common properties, but does 
not give a general analogue of the Shuffle Theorem. Bergeron and Préville-Ratelle 
[8] specifically examine the case of three sets of variables with a simultaneous G,, 
action and conjecture a natural analogue of the Shuffle Theorem, although without 
proposing a necessary third statistic. Since the closely related Hilbert scheme of 
points in the plane is no longer smooth for more than two sets of variables, Haiman 
[33] comments that the algebraic geometric techniques used to explore the diagonal 
harmonics do not readily generalize to the case of additional sets of variables. 


Acknowledgements The author would like to express her gratitude for the many helpful remarks 
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On Positivity of Ehrhart Polynomials ®) 


Check for 
updates 


Fu Liu 


Abstract Ehrhart discovered that the function that counts the number of lattice 
points in dilations of an integral polytope is a polynomial. We call the coefficients 
of this polynomial Ehrhart coefficients and say a polytope is Ehrhart positive if 
all Ehrhart coefficients are positive (which is not true for all integral polytopes). 
The main purpose of this chapter is to survey interesting families of polytopes that 
are known to be Ehrhart positive and discuss the reasons from which their Ehrhart 
positivity follows. We also include examples of polytopes that have negative Ehrhart 
coefficients and polytopes that are conjectured to be Ehrhart positive and pose a few 
relevant questions. 


1 Introduction 


A polyhedron in the D-dimensional Euclidean space R? is the solution set of a finite 
set of linear inequalities: 


D 
P= x eR? : Sai jx; < bi forie r+, 
j=l 


where aj,; € R, bj € R, and J is a finite set of indices. A polytope is a bounded 
polyhedron. Equivalently, a polytope in R? can also be defined as the convex hull 
of finitely many points in R?. We assume readers are familiar with basic definitions 
such as faces and dimensions of polytopes as presented in [108]. In this paper, the 
letter d usually denotes the dimension of a polytope and D denotes the dimension 
of the ambient space. For majority of the examples presented here, we either have 
d=Dord=D~-1. 
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A lattice point or an integral point is a point in Z”. Counting lattice points inside 
polytopes is a fundamental and useful step in many mathematical analyses. A lot of 
combinatorial structures can be counted as lattice points of polytopes. For example, 
matchings on graphs [63], t-designs [74], (semi-)magic squares [10, Chapter 6], and 
linear extensions of posets [102] are all of this form. Counting lattice points not 
only appears in combinatorial problems, it also appears, for instance, in the context 
of representation theory [52, 89], algebraic geometry [37], statistics [34, 36], and 
number theory [7, 75]. 

One approach to study the question of computing the number of lattice points in 
a polytope P is to consider a more general counting problem: For any nonnegative 
integer t, lett P := {tx : x © P} be the fthdilationof P, and then consider the 
function 

i(P,t) = |tP NZ? |, 


which counts the number of lattice points in tP. We say two polytopes P and Q 
are unimodularly equivalent’ if there exists an affine transformation from the affine 
hull aff(P) of P to the affine hull aff(Q) of Q that induces a bijection from lattice 
points in aff(P) to lattice points in aff(Q). Such an affine transformation is called 
a unimodular transformation. It is easy to see that if two polytopes P and Q are 
unimodularly equivalent, then i(P, t) = i(Q, ft). 

An integral polytope (or a lattice polytope) is a polytope whose vertices are lattice 
points. In the 1960s, Eugéne Ehrhart [35] discovered that the function i(P, t) has 
nice properties when P is an integral polytope. 


Theorem 1.1 (Ehrhart). For any integral d-polytope P, the function i(P,t) is 
always a polynomial (with real coefficients) of degree d int. 


Thus, we call i(P, t) the Ehrhart polynomial of an integral polytope P, and the 
coefficients of i(P, t) the Ehrhart coefficients of P. Note that Ehrhart’s theorem can 
be extended to rational polytopes with the concept of a quasi-polynomial; however, 
we will focus on integral polytopes in this chapter. 

There is much work on the Ehrhart coefficients of integral polytopes. In the 1990s, 
many people studied the problem of counting lattice points inside integral (or more 
generally rational) polytopes [17, 22, 50, 80, 85] by using the theory of toric varieties. 
Although explicit formulas for coefficients of Ehrhart polynomials can be deduced 
from these results, they are often quite complicated. Only three coefficients of i(P, t) 
have simple known forms for arbitrary integral polytopes P: The leading coefficient 
is equal to the normalized volume of P, the second coefficient is one half of the sum 
of the normalized volumes of facets, and the constant term is always 1. 

Although these three coefficients can be described in terms of volumes (recall 1 
is the normalized volume of a point) and thus are positive, it is not true that all the 
coefficients of i(P, t) are positive. (The first counterexample comes up in dimension 
3 known as the Reeve tetrahedron; see Sect.4.1.) We say a polytope has Ehrhart 


1 Unimodular equivalence is sometimes called integral equivalence, e.g., in [79]. 
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positivity or is Ehrhart positive if it has positive Ehrhart coefficients. It is natural to 
ask the following question: 


Question 1.2. Which families of integral polytopes have Ehrhart positivity? 


This turns out to be a challenging question. Even though multiple families of 
polytopes have been shown to be Ehrhart positive in the literature, the techniques 
involved are (almost) all different. In Sect.2, we will survey families of polytopes 
with the Ehrhart positivity property, discussing different reasons why they have this 
property. In particular, as a consequence of the techniques discussed in Sect. 2.4, 
one can show that any combinatorial type of rational polytopes can be realized 
as an integral polytope that is Ehrhart positive (see Theorem 2.4.8). This result 
indicates that Ehrhart positivity is not a combinatorial property. Therefore, it is 
desirable to find more geometric methods to prove Ehrhart positivity. In Sect.3, we 
introduce such a tool called McMullen’s formula, which we use to give a refinement 
of Ehrhart positivity, called a-positivity. We then use this tool to attack the Ehrhart 
positivity conjecture of “generalized permutohedra,” a family of polytopes introduced 
by Postnikov [82], and report partial progress on this conjecture. In Sect.4, we 
include negative results on Question 1.2, presenting examples with negative Ehrhart 
coefficients. In particular, we will discuss progress on a question asked and studied 
by Hibi, Higashitani, Tsuchiya, and Yoshida [47] on all possible sign patterns of 
Ehrhart coefficients (see Sect.4.2). Note that this question can be considered to be 
a refinement of Question 1.2. Finally, in Sect.5, we include various conjectures on 
Ehrhart positivity and pose related questions. 

We finish our introduction with the following remark on the coefficients of the 
h*- polynomial, which is closely related to the Ehrhart polynomials. 

Remark on h*-Vector. One method of proving Ehrhart’s theorem (Theorem 1.1) is 
by considering the Ehrhart series of a d-dimensional integral polytope P: 


Ebr p(z) := Ss i(P, t)z’. 


t>0 


It turns out that Theorem 1.1 is equivalent to the existence of a polynomial h%,(z) of 
degree at most d such that h7,(1) 4 0 and 


hp) 
Ehr p(z) = a-pa 
See [94, Chapter 4] for a statement for the above equivalence result and a proof for 
Ehrhart’s theorem. 

We call h,(z) the h*-polynomial of P, and the vector (hj, hj, ..., h%), where 
h* is the coefficient of Zin h*,(z), the h*-vector of P. One can recover the Ehrhart 
polynomial of a d-dimensional integral polytope P easily from its h*-vector: 
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d 


iP. = mre’), (1.1) 


j=0 


It is a well-known result due to Stanley that the entries in h*-vectors are all nonneg- 
ative integers [93] in contrast to the fact that Ehrhart coefficients could be negative. 
AS a consequence, positivity is not such an interesting question for h*-polynomials. 
Nevertheless, active research has been conducted in other directions. 

The most natural question probably is: For each d, can we give a complete charac- 
terization for all possible h*-vector of d-dimensional integral polytopes? For d = 2, 
the answer was first provided in 1976 by Scott [90] known as Scott’s condition. 
However, for d > 3, the question is wide open. A lot of work has been done in 
the literature on searching for inequalities and equalities satisfied by h*-vectors. 
Most of them were discovered by Hibi [43-46] and Stanley [93, 99] in the 1990s 
using commutative algebra and combinatorial methods. In 2009, Stapledon [103] 
contributes more inequalities using the idea of degree and codegree of a polytope. 
Known equalities on h*-vectors include 


d 
hi =d!Vol(P), ho =1, hp =i(P)-@+), Ay = |relint(P) Z|. 
i=0 


(1.2) 
Please see [9, 103] for lists of known inequalities. Recently, instead of focusing on 
inequalities satisfied by all polytopes, much work has been done on finding inequal- 
ities for polytopes under certain constraints. For example, Treutlein [105] shows 
that the necessary statement of Scott’s condition holds for any integral polytope 
whose h*-polynomial is of degree at most 2, i.e., h¥ = 0 for alli > 2. Most recently, 
Balletti and Higashitani [5] improve the result further to any integral polytope whose 
h*-polynomial satisfies h} = 0. 

Another question that comes up a lot in the context of h*-vector is the uni- 
modality question. A sequence of real numbers co, cj, ..., Cq is unimodal if there 
exists 0 < j < dsuchthat cg <c) <+-- <cj > ¢j41 > +++ = cq. It is well known 
that a nonnegative sequence is unimodal if it has “no internal zeros” and is “log- 
concave,” and furthermore, log-concavity follows from another property called “real- 
rootedness.” Please see surveys by Stanley [98] and Brenti [16] on log-concave and 
unimodal sequences, and a survey by Brandén [13] with a more general discussion 
on unimodality, log-concavity, and real-rootedness. Recently, Braun [14] wrote a 
survey on unimodality problem of the *-vector of integral polytopes, discussing a 
wide range of tools (including but not limited to the techniques mentioned in the 
aforementioned surveys) to attack this problem. Finally, we would like to remark 
that even though Ehrhart coefficients and h*-vectors are related by (1.1), there is no 
general implication between Ehrhart positivity and h*-unimodality [60]. 
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2 Polytopes with Ehrhart Positivity 


In the literature, there are multiple interesting families of polytopes shown to be 
Ehrhart positive using very different techniques. In this section, we put together a 
collection of such families, separating them into four categories based on the reasons 
why they are Ehrhart positive. However, we make no attempt to give a comprehensive 
account of all families with this property. We also note that as the leading coefficient 
of i(P,t) is the volume of P, one can often extract a formula for volume from 
descriptions for Ehrhart polynomials we give below. However, we will focus only 
on results on Ehrhart polynomials here and omit related formulas for volumes. 

In this chapter, we use bold letters to denote both vectors and points in R?. 
For example, e; denotes both the ith vector in the standard basis and the point 
(0,...,0, 1,0,...,0) where | is in the 7th position. 

For convenience, we use N to denote the set of nonnegative integers, and P the 
set of positive integers. 


2.1 Products of Positive Linear Polynomials 


In this part, we present families of polytopes whose Ehrhart polynomials can be 
described explicitly, which can be shown to have positive coefficients using the 
following naive lemma. 


Lemma 2.1.1. Suppose a polynomial f (t) is either 


(a) a product of linear polynomials with positive coefficients, or 
(b) asum of products of linear polynomials with positive coefficients. 


Then f (t) has positive coefficients. 


We start with the two simplest families of polytopes: unit cubes and standard 
simplices, whose Ehrhart polynomials fit into situation (a) of Lemma 2.1.1, and thus 
Ehrhart positivity follows. As these are the first examples of Ehrhart polynomials in 
this chapter, and the computations are straightforward, we include all the details. For 
most of the remaining examples we discuss in this paper, we only state the results 
without providing detailed proofs. 


2.1.1 Unit Cubes 


The d-dimensional unit cube, denoted by Lg, is the convex hull of all points in R? 
with coordinates in {0, 1}, ie., 


a = conv{x = (x1, %,...,%y) € RY : x; =Oorl fori = 1,2,...,d}. 
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It is easy to verify that the unit cube is the solution set to the following linear system 
of inequalities: 


ag = {x = (x1, %2,...,X¢) ER? : O< x; < 1 fori =1,2,...,d}, 


Then for any t € N, 


tOy = {x = (41, %0,..-,%¢) € RY : O< x; <t fori =1,2,...,d}. 


Thus, 


tOg NZ? = {x = (x1, x0,...,x¢) EZ? : O< x; <t fori =1,2,..., d}. 


For each 7, the number of integers x; such that 0 < x; <tist-+ 1. Thus, 


i(Qa,t) = (t+ 1%. 


2.1.2 Standard Simplices 


The d-dimensional standard simplex, denoted by Ag, is the convex hull of all the 
elements in the standard basis e;, e2,..., @g41 of R!, 


Aq := conv{e;, @2,..., @g44}. 


One checks that Aq can also be defined by the following linear system: 


d+1 
be = 1, andx; >0 fori=1,2,...,d+1. 
j=l 

Hence, for any ¢t € N, 


d+1 
tAg= x = (x1, %2,...,%¢41) € RO! . ixeast and x; > Ofori =1,2,...,d+17, 
j=l 


and 
d+l 
(My NZY =e = Gyre sineeeyeZ 2 Y xpot, andy 20 fort =1,2,,.0; d+ 
j=l 


Hence, i(Aq, t) counts the number of nonnegative integer solutions to 


XptxXg+-++ + X41 =. 
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This is a classic combinatorial problem which is the same as counting the number of 
weak compositions of f into d + | parts (see [94, Page 18]), and the answer is given 


by 
: t+d 
i(Ag, t) = ( d ) 


As we mentioned above, Ehrhart positivity of unit cubes and standard simplices 
follows from situation (a) of Lemma 2.1.1. Next, we present two families of examples 
with Ehrhart polynomials in the form of situation (b) of Lemma 2.1.1. 


2.1.3 Pitman-Stanley Polytopes 


Let a = (aj,..., aq) € N¢. The following polytope is introduced and studied by 
Pitman and Stanley [79]: 


PSq(a) := xeR! : x, >0and )oxj< Soai, fori =1,2,...,d}, 


j=l j=l 


and hence we call it a Pitman-Stanley polytope. 
Pitman and Stanley gave an explicit formula [79, Formula (33)] for computing the 
number of lattice points in PS,(a), from which a formula for the Ehrhart polynomial 


of PSq(a) immediately follows. Recall ( ) = aces: 


x 
y 


Theorem 2.1.2 (Pitman-Stanley). Let 


d k 
Ig [i= ere ia) EN? : \Oij =d, and Qi; ehirb=.t tf, 


j=l j=l 


Then the Ehrhart polynomial of PSa(a) is given by 


d 
i(PSa(a),) = >> (“> ')) I] ((“')) ; (2.1) 
k=2 


tely 


For each i, both ( ce ) and ( fs : ) are products of linear polynomials in ¢ with 
positive coefficients, so it follows from Lemma 2.1.1/(b) that any Pitman-Stanley 
polytope PS,(a) is Ehrhart positive. 

Pitman-Stanley polytopes are contained in two different more general families 
of polytopes: flow polytopes and generalized permutohedra. For each of these two 
bigger families of polytopes, formulas for Ehrhart polynomials of some subfamily 
have been derived, generalizing Formula (2.1). We present results on flow polytopes 
in the next part below, while the results on generalized permutohedra are postponed 
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to Sect.3.1.2 as part of a general discussion on the Ehrhart positivity conjecture of 
generalized permutohedra in Sect. 3. 


2.1.4 Subfamilies of Flow Polytopes 


Let G be a (loopless) directed acyclic connected graph on [n + 1] = {1,2,...,2+ 
1} such that each edge {i, j} with i < j is always directed from i to 7. Hence, we 
denote the edge by (i, j) to indicate the orientation. For any a = (a), d2,..., Gn) € 
N”, we associate to it another vector 


a:= (01. --.40-Yoa) (2.2) 


An a-flow on G is a vector f = (f(e))eex(G) € (Rx) such that for i = 
1,2,...,n, we have 


Y fet+a= Yo fe, 


e=(g,1)EE(G) e=(i, /)EE(G) 


that is, the netflow at vertex i is a;. Note these conditions imply that the netflow at 
vertex n + lis — )~"_, a;. The flow polytope Fg (a) associated to G and the integer 
netflow a is the set of all a-flows f on G. 


Example 2.1.3. Let G7° be the graph on [d + 1] with edge set 
{G,i+1),@,d+1) :i=1,2,...,d}. 


Baldoni and Vergne [3, Example 16] show that Fgps (a) is unimodularly equivalent 
to the Pitman-Stanley polytope PS,(a). 


For each edge e = (i, j) of G, we associate to it the positive type A, root a(e) = 
a(i, j) = e; — e;. For any b € Z”"', the Kostant partition function KP g evaluated 
at b is 


KP o(b) :=#4) f =(f)ece EN? : Yo fale) =b 
ec E(G) 
It is straightforward to verify that for a € N”, 
KP¢@) = |Fe@NZFO, 


i.e., K’PG(a) counts the number of lattice points in the flow polytope Fg (a). In the 
literature, various groups of people [3, 55, 70, 84] obtained formulas for Kostant 
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partition functions or equivalently the number of lattice points in flow polytopes. As 
a consequence, we can easily obtain formulas for the Ehrhart polynomial of Fg (a). 


Theorem 2.1.4 (Lidskii, Postnikov-Stanley, Baldoni-Vergne, Mészaros-Morales). 
Suppose G is a connected graph on the vertex set [n + 1], with m edges directed 
i — j ifi < j, and with at least one outgoing edge at vertex i fori = 1,...,n. Let 
out, (and ing, respectively) denote the outdegree (and the indegree, respectively) of 
vertex k in G minus 1. 

Then for any a = (a,,...,4n) € N", the Ehrhart polynomial of Fg(a) is given 
by 


i(F¢(@), t) 
n 
ayt + out; : ; : 
-DIT( ; ) -KPG(i — out;, jz — Outs, ..., jn — Out,, 0), (2.3) 
i; k= Jk 
J k=l 
n . 
ayt — Ing . . ; 
= 1] (( ; )) -KPg (ji — out), jp — outy,..., jn — Out, 0), (2.4) 
k=l Jk 
j k= 
where each summation is over all weak compositions J = (ji,..-, jn) of m — n that 
are > (out,,..., out,) in dominance order. 


We remark that Lidskii [55] gives a formula for computing Kostant partition func- 
tions associated to the complete graph K,,,;, which yields Formula (2.3) above with 
G = K,,. Postnikov and Stanley [84, unpublished] were the first to discover For- 
mula (2.3) for arbitrary graphs G using the Elliott-MacMahon algorithm. Baldoni 
and Vergne [3] give a proof for both formulas in Theorem 2.1.4 using residue com- 
putation. Most recently, Mészaros and Morales [70] recover Baldoni-Vergne’s result 
by extending ideas of Postnikov and Stanley on the Elliott-MacMahon algorithm and 
polytopal subdivisions of flow polytopes. 

Formula (2.4) is useful in obtaining positivity results since 


((“" — ing )) _ (“ — ing + jp — ') 
Jk Jk 


is a product of linear polynomials in ¢ with positive coefficients as long as ing = 0 or 
—1. Also, note that CP¢(j; — out, j2 — outz,..., j, — out,, 0) is nonnegative and 
i(Fg(a), t) € 0. The following result immediately follows from Lemma 2.1.1/(b). 


Corollary 2.1.5. Assume the hypotheses of Theorem 2.1.4. Assume further that for 
each vertex i € [n] = {1,2,...,n}, the indegree of i is either 0 or 1. Then the flow 
polytope Fg(a) is Ehrhart positive. 


We remark that the graph Go defined in Example 2.1.3 satisfies the hypothesis 
of the above corollary. Hence, Ehrhart positivity of the Pitman-Stanley polytope is a 
special case of Corollary 2.1.5. 
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2.2 Roots with Negative Real Parts 


In this part, we show examples with Ehrhart positivity using the following lemma. 
We use Ste(z) to denote the real part of a complex number z. 


Lemma 2.2.1. Let p(t) be a polynomial in t with real coefficients. If the real part 
Re(r) is negative for every rootr of p(t), then all the coefficients of p(t) are positive. 


Proof. Leta > 0. If —a < 0 is areal root of p(t), then t + a is a factor of p(t). If 
—a + bi is acomplex root of p(t) for some b € R, then —a — bi must be a root of 
p(t) as well, which implies that 


(4+e=—biG a+b) = +26 a" £P) 


is a factor of p(t). Hence, p(t) is a product of factors with positive coefficients. Thus, 
our conclusion follows. 


We say that a polynomial (with real coefficients) is negative-real-part-rooted or 
NRPR if all of its roots have negative real parts. The above lemma implies that if 
i(P,t) is NRPR, then P is Ehrhart positive. Ehrhart polynomials of unit cubes and 
standard simplices are all trivially NRPR, as they factor into linear polynomials 
with positive real coefficients. Hence, we would like to rule them out and are only 
interested in examples of Ehrhart polynomials that are nontrivially NRPR. 

It turns out that the following theorem which establishes a connection between 
roots of the h*-polynomial and roots of the Ehrhart polynomial of a polytope is very 
useful. 


Theorem 2.2.2 ((88], Theorem 3.2 of [101]). Let P be a d-dimensional integral 
polytope, let k be the degree of the polynomial h*,(z) (so0 < k < d), and suppose 
that every root of h*,(z) lies on the circle {z € C : |z| = 1} in the complex plane. 
Then there exists a polynomial f (t) of degree k such that 


d-k 
i(P,t) = f()- | ][@+d, 
i=l 
and every root of f (t) has real part —(1 + (d —k))/2. 


We say a polytope P is h*-unit-circle-rooted if the h*-polynomial h7,(z) of P 
has all of its roots on the unit circle of the complex plane. Below we introduce three 
families of polytopes and show that each polytope P in these families is h*-unit- 
circle-rooted. Therefore, Ehrhart positivity for these families follows from Theorem 
2.2.2 and Lemma 2.2.1. 


2.2.1. Cross-Polytopes 


The d-dimensional cross-polytope, denoted by },, is a polytope in R@ defined by 
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Og := conv{te), te2,..., ey}, 


or equivalently by the following linear system: 


Ax, $x. +--+ tug <1. 


Hence, i(Qg, t) counts the number of integer solutions to 
|x1| + [x2] +--+ [xal St. 


Counting the number of integer solutions with exactly k nonzero x;’s for k = 
0,1,2,...,d, we obtain that 


q d t 
i(a,t) = 2( )( 
oe k]\k 


Unfortunately, it is not clear whether the above expression expands positively in 
powers of t. We compute its Ehrhart series instead. First, notice that i(Qa, t) counts 
the number of integer solutions to 


xi] + [x2] +--+ [xal ty =F. 


Hence, 


iat) = >> fa) f(@)... f(aa) gb), 


where the summation is over all weak compositions (a),..., aa, b) of t intod + 1 
parts, g(b) = 1 forallb > Oand f(a) = lifa = Oand f(a) = 2ifa > 0. Therefore, 


: t ‘ Qj 1+2z 1 (1+z)4 
\ i@adz “T(E ree") ye = (4) ‘ = = (— zt!’ 


t>0 i=1 \aj>0 b>0 


Thus, (1 + z)“ is the h*-polynomial of the cross-polytope 4. Hence, ©q is h*-unit- 
circle-rooted and thus is Ehrhart positive. 


2.2.2 Certain Families of A (1,4) 


Let q = (41, q2,---,qa) € P@ be a sequence of positive integers. For each such a 
vector q, we define a simplex 


d 
A(i,gq) ‘= CONV } 1, €2,..., dg, — ) ae : 
i=l 
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In [29], Conrads studied simplices of this form and showed that A(;,q) is reflexive if 


and only if 
d 


qi divides1+ 5° qj, fori =1,2,...,d. 
j=l 
Recently, Braun, Davis, and Solus studied A (,q) in their investigation of a conjecture 


by Hibi and Ohsugi, and they provided a number-theoretic characterization of the 
h*-polynomial of Aq.q) [15, Theorem 2.5]. 


Theorem 2.2.3 (Braun-Davis-Solus). The h*-polynomial of Ac, ,q) is 
M+gat +a 
Wikpaeys So ale, (2.5) 


b=0 


where 


w(b) = b 3 qib 
of Lisgose ge) 


Formula (2.5) allows us to compute the h*-polynomial for Aig) with special 
choices of g easily. We give two examples below in which A; q) satisfies the hypoth- 
esis of Theorem 2.2.2 with k = d. 


Example 2.2.4 (Standard reflexive simplices). If we chooseg =1= (1,1,...,D € 
P?, then A i,q) is the d-dimensional standard reflexive simplex. Note that in this case, 
we have thatg; + g2 +--:+ qa = d. Furthermore, foreachb € {0,1,2,...,d}, one 
can verify that w(b) = b. Hence, 


d 
h* (Aaay, 2) =)oe alert ote tel. 
b=0 


Example 2.2.5 (Payne’s construction). In [78], Payne constructed reflexive sim- 
plices that do not have unimodal h*-vectors. His construction is equivalent to the 
simplices A(,,4) with the following choices of g : Givenr > 0,s > 3andk >r-+2, 
letd =r-+sk and 


q = (@1,9,..-,44) =(1,1,..., 1,5, 5,...,5). 
——— eS 
sk—1 times r+l times 


Applying Theorem 2.2.3, one can obtain 


h* (Aa. 2) = At et eete +28 PQ 4 24 4-427), 
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Polytopes in both Examples 2.2.4 and 2.2.5 are h*-unit-circle-rooted. Hence, they 
are Ehrhart positive. We remark that the Ehrhart positivity of Ay,,) considered in 
Example 2.2.5 was first proved by the author and Solus [60, Theorem 3.2]. 


2.2.3 One Family of Order Polytopes 


Given a finite poset (partially ordered set) P, the order polytope, denoted by O(P), 
is the collection of functions x € R” satisfying 


e 0<x, <1, foralla € P, and 
eX, < xp, ifa<binP. 


The order polytope O(P) was first defined and studied by Stanley [102]. Here we 
consider a family of order polytopes constructed from a certain family of posets. 

Let P; be the ordinal sum of k copies of 2 element antichains; equivalently, P; is 
the poset on the 2k-element set {a;,; : i = 1,2 and j = 1,2,...,k} satisfying 


qj,; < aj; if and only if j < j’ or Gi, j) = (i, j’). 


For any t € N, the ¢th dilation rO(P;) of O(P,) is the collection of x = (xj; : i= 
1,2 andj = 1,2,...,k)€ R* satisfying 


O< x, <6, and Ms keg it 7 < gz 


Hence, i(O(P,), t) counts the number of integer solutions x satisfying the above two 
conditions. Note that each solution gives a weak composition (y1, Z1, V2, --- + Ves Zks 
Ye+1) Of t into 2k + 1 parts, where 


Yj =min(x,;, X2,;) = max(x1,j-1, X2,j-1), for j => 1,2; ates vk+ 1, 


gj =max(x1,;, X2,;) = min(x1,;, X2,;) = 1X1, = x2, j|, for j = 1, 2, sees k, 


and by convention let max(x1,0, X2,0) = O and min(x1 441, X2,441) = t. Thus, 


i(O(Pk). 1) = > gov fe 902) f Ga) --. f KIO: 


where the summation is over all weak compositions of ¢ into 2k + 1 parts, g(y) = 1 
for all y > 0, and f(z) = lif z= Oand f(z) = 2 if z > 0. Therefore, similar to the 
calculation for cross-polytopes, we obtain 


(1 + z)* 
al _ z)2k+1 i 


DOP), Nz = 


t>0 


Thus, the h*-polynomial of O(P;) is (1 + z)*. By Lemma 2.2.1 and Theorem 2.2.2, 
the order polytope O(P;,) is Ehrhart positive. 
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Remark 2.2.6. Stanley also defined a “chain order” polytope C(P) for each poset 
P [102, Definition 2.1] and showed that C(P) is unimodularly equivalent to O(P) 
[102, Theorem 3.2/(b)], from which it follows that i(C(P), t) = i(O(P), t). 

Therefore, the conclusions we draw above for the order polytope O(P,) all hold 
for the chain polytope C(P;). 


It turns out that the polytopes studied in Sects.2.2.1 and 2.2.2 are “reflexive” 
polytopes, and the order polytopes studied in Sect. 2.2.3 are “Gorenstein” polytopes. 
These are not coincidences as we will discuss below. 


Connection to Reflexivity and Gorensteinness. An integral polytope P in R? is 
reflexive (up to lattice translation) if the origin is in the interior of P and its dual 


PY :={y € (R”)* : (y,x) <1Vx € P} 


is also an integral polytope, where (R”)* is the dual space of R”. 

It follows from the Macdonald Reciprocity Theorem [64] that if an integral poly- 
tope P is reflexive, then the roots of i(P, t) are symmetrically distributed with respect 
to the line {z € C : Re(z) = —1/2} in the complex plane. Bey, Henk, and Wills 
show that the converse is true if we include polytopes that are unimodularly equiv- 
alent to reflexive polytopes [12, Proposition 1.8]. Recently, Hegediis, Higashitani, 
and Kasprzyk, in their study of roots of Ehrhart polynomials of reflexive polytopes, 
give the following result [41, Lemma 1.2]. 


Lemma 2.2.7 (Hegediis-Higashitani-Kasprzyk). A d-dimensional integral polytope 
P is reflexive (up to unimodular transformation) if and only if the summation of the 
d roots of i(P, t) equals to —d/2. 


Reflexive polytopes are special cases of a more general family of polytopes: 
Gorenstein polytopes. Recall that the codegree of P is defined to be 


codeg(P) := dim(P) + 1 — deg (h3(z)) . 


Itis (again) aconsequence of the Macdonald Reciprocity Theorem [64] that codeg(P) 
is the smallest positive integer s such that sP contains a lattice point in its interior 
(see, e.g., [48]). A Gorenstein polytope is an integral polytope P of codegree s such 
that s P is a reflexive polytope. The work [95] of Stanley gives a nice characterization 
for the h*-polynomials of Gorenstein polytopes: A d-dimensional integral polytope 
P is a Gorenstein polytope if and only if its h*-polynomial is symmetric; that is, if 
h3(z) = ey h*z' with ht A 0, then h*¥ = h#_, fori = 0, 1,2,..., k. Using this, 
one can easily see that all the examples discussed in Sects. 2.2. 1—2.2.3 are Gorenstein 
polytopes. 
We now restate Lemma 2.2.7 in terms of Gorenstein polytopes. 
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Lemma 2.2.8. A d-dimensional integral polytope P is Gorenstein (up to unimodular 
transformation) if and only if the summation of the d roots of i(P, t) equals to —sd /2 
for some positive integer s. 

Furthermore, if the above condition holds, the integer s is the codegree of P. 


Proof. The conclusion of the lemma follows from the observation that a number fo 
is a root of i(P, t) if and only if fo/s is a root of i(sP, t). 


Corollary 2.2.9. Suppose P is a d-dimensional polytope that is h*-unit-circle- 
rooted. Then P is a Gorenstein polytope (up to unimodular transformation). 
Moreover, if the degree of the h*-polynomial h’,(z) is d, then P is reflexive. 


Proof: Let k be the degree of h7,(z). By Theorem2.2.2, among all the roots of 
i(P,t), k of them have real parts —(1 + (d — k))/2, and the other (d — k) roots are 

1,-2,...,-(d —k). Asi(P, t) is a polynomial with real coefficients, the sum of 
roots of i(P, t) is the sum of the real parts of roots of i(P, t), which is 


d—k 
—(1+ (d—k)/2-k+0(-i) = — Sad —k+1). 


i=1 


Then the conclusions follow from Lemma 2.2.8. 


Therefore, when an integral polytope P is h*-unit-circle-rooted, we not only get 
Ehrhart positivity for P but can also conclude that P is a Gorenstein polytope of 
codegree d — k + 1, where k is the degree of the h*-polynomial of P. 


2.3 Coefficients with Combinatorial Meanings 


2.3.1 Zonotopes 


In this part, we introduce a special family of polytopes, zonotopes, whose Ehrhart 
coefficients can be described combinatorially. As a consequence, Ehrhart coefficients 
of a zonotope are not only positive but also positive integers. 

The Minkowski sum of two polytopes (or sets) P and Q is 


P+Q:={x+y:xeEP, yeQ}. 
Let v1, v2,..., ¥, € Z? beaset of integer vectors. The zonotope Z(v1, V2,..., Un) 


associated with this set of vectors is the Minkowski sum of intervals [0, v;], where 
[0, v;] is the line segment from the origin to v;. Hence, 


Z(v1,...,0n) = D0, v=} > cn; 1 OSG < Hoh 1 2evaf 


i=l i=l 
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In [92, Theorem 2.2], Stanley gives a combinatorial description for the Ehrhart coef- 
ficients of zonotopes. 


Theorem 2.3.1 (Stanley). The coefficient of t* in i(Z(v1,..., Vn), t) is equal to 
pRe 
x 


where X ranges over all linearly independent k-subsets of (v1, ..., Un}, and h(X) is 
the greatest common divisor of all k x k minors of the matrix whose column vectors 
are elements of X. 


The main ingredient for the proof of the above theorem is that Z(v,,..., v,) can 
be written as a disjoint union of half-open parallelepiped Cy ranging over all linearly 
independent subsets X = {v;,,..., v;,} Of {v,,..., U,}, where Cy is generated by 
€|0;,,..-, €;V;, for certain choices of €],..., €, € {—I, 1}. (see [92, Lemma 2.1].) 
The theorem then follows from the fact that the number of lattice points in the half- 
open parallelepiped Cy is the volume of Cx, which can be calculated by h(X). 

The simplest examples of zonotopes are unit cubes considered in Sect.2.1.1. We 
may recover the Ehrhart polynomial of a unit cube using Theorem 2.3.1. However, a 
more interesting example is the regular permutohedron. 


Example 2.3.2. The regular permutohedron, denoted by Tq, is the convex hull of 
all permutations in Gy4,; that is, 


Tg := conv{(o(1), o(2),...,0(d + 1) € RO! so € Gay}. 


It is straightforward to check that I, is a translation of the zonotope 


> [0, e; — ej]. 


l<i<j<d+1 


For any subset X of ®y := {e; —e; : 1 <i < j <d-+1}, we let Gy be the graph 
on vertex set [d + 1] and {i, j} (with i < j) is an edge if and only if e; — e; € X. 
Then it follows from matroid theory that X is linearly independent if and only if Gx 
is a forest on [d + 1]. (Recall that a forest is a collection of trees or equivalently is 
an acyclic graph.) Furthermore, if X is linearly independent, then Gy is a forest of 
d+1— |X| trees, and h(X) (described in Theorem 2.3.1) is 1. 

Therefore, we conclude that the coefficient of t* in i(My, t) counts the number of 
forests on [d + 1] that contain exactly d + 1 — k trees. Therefore, we can compute, 
for example, 

i (ly, 2) = 167? +1577 +60 + 1. 


On Positivity of Ehrhart Polynomials 205 
2.3.2 Positivity of a Generalized Ehrhart Polynomial 


The polynomial we discuss in this part is not exactly an Ehrhart polynomial. However, 
it is closely related to the result on zonotopes we have presented in the last part, and 
thus is included here. Galashin, Hopkins, McConville, and Postnikoy, in their study 
of root system chip firing [38], considered the following lattice points counting 
problem: Given an integral polytope P and a set of integer vectors v;, U2,..., Up, 
describe the number of lattice point in 


P+to,ptouot+---+v,= P+ Z(v1,..., 0). 


Extending Stanley’s idea of decomposing zonotopes into half-open parallelepiped, 
they show [38, Proof of Theorem 16.1] that P + Z(v,,...,v,) can be written as 
disjoint union of sets in the form of F + Cx where F is an open face of P and Cx isa 
half-open parallelepiped determined by a linearly independent set X of {v;,..., v,}. 


Theorem 2.3.3 (Galashin-Hopkins-McConville-Postnikov). Suppose P is an inte- 
gral polytope in RP and v4,...,U, € Z? is a set of integer vectors. Let Z = 
Z(v,,...,U,). For any t = (t,...,t) € Ny, we define tZ = Z(t v1, ..., t,Vn)- 
Then there exists a polynomial L(t) = L(t), ..., t,) inn variables with nonnegative 
integer coefficients such that |(P +tZ)Z?| = Lit). 

In particular, if we take t = (t,t,...,t), then 


\(P+tZ)NZ?| = |(P+tZ)NZ?| = Lit,t,...,1) 


is a polynomial in t of degree dim(Z) with positive integer coefficients. 


Note that the second part of the above theorem was not explicitly stated in [38, 
Theorem 16.1], but it was a consequence of the techniques used in its proof. 

One sees that if we choose P to be the origin, then the above theorem recovers 
the Ehrhart positivity of zonotopes. However, in contrast with Stanley’s results, no 
explicit formulas were given in [38] for the positive/nonnegative integer coefficients 
asserted in Theorem 2.3.3. Recently, Hopkins and Postnikov [49] analyzed tech- 
niques used in [38] further and provided the desired explicit formula, completing the 
generalization of Theorem 2.3.1. 


Theorem 2.3.4 (Hopkins-Postnikov). The homogeneous degree k part of the poly- 
nomial L(t) assumed by Theorem 2.3.3 is given by 


»~ | quot y (P) N quoty(Z?)| - h(X) - I] f 
X 


vjieX 


where X ranges over all linearly independent k-subsets of {v1,..., Un}, Quoty : 
R? — R?/spang(X) is the canonical quotient map, and h(X) is the greatest com- 
mon divisor of all k x k minors of the matrix whose column vectors are elements 
of X. 
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2.4 Higher Integrality Conditions 


In this part, we will introduce families of polytopes whose Ehrhart coefficients are 
always volumes of certain projections of the original polytopes and are hence positive. 
2.4.1. Cyclic Polytopes 


We start with a well-known family of polytopes: cyclic polytopes. The moment curve 
in R¢ is defined by 


wi R2> RY,x bh gu) = (u,w’,...,u%). 
Let U = {u,...,Un}< be a linear ordered set. Then the cyclic polytope Cq(U) = 
Ca(uj,...,Un) 1s the convex hull of n > d distinct points vy(4;), 1 <i <n, on the 


moment curve: 
Ca(U) := conv{vg (uy), Va (tz), -.-, YgUn)}- 


Cyclic polytopes form an interesting family of polytopes. For instance, its facets are 
determined by the Gale evenness condition [108, Theorem 0.7], and the number of 
i-dimensional faces of Cg(U) (where |U| = 7) is the upper bound for the number of 
i-dimensional faces of all d-dimensional polytopes with n vertices [66]. 

The following theorem on the Ehrhart polynomial of integral cyclic polytopes 
was initially conjectured in [9] by Beck, De Loera, Develin, Pfeifle, and Stanley and 
then proved in [57] by the author. 


Theorem 2.4.1 (L.). For any d-dimensional integral cyclic polytope P = Cqg(U) C 
R¢, we have that 


d 
i(P, t) = Volg(P)t4 + i(1(P), t) = Y> Vol Commie 0) eae (2.6) 
k=0 


where r4—®) : R4 — R* is the map that ignores the last d — k coordinates of a point, 
and Vol,(Q) is the volume of Q in the k-dimensional space R*. 


The first step of the proof is to reduce the problem to simplices by using trian- 
gulations. For the simplex case, we consider the set obtained by removing the lower 
envelope of Cy(U) (with |U| = d+ 1), and we decompose this set into d! signed 
(convex) half-open sets S,, each of which corresponds to a permutation o in the sym- 
metric group Gg. One important feature of this decomposition is that the number of 
lattice points in each piece S, can be expressed in a simple formula involving the 
permutation o, which makes it possible to compute the summation of all d! terms. 
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(2,2,2) a 


(0,0,0) 


(4,0,0) 


Fig. 1 Examples of higher integrality conditions 


2.4.2 k-Integral Polytopes 


Since the work in [57], the author generalized the family of integral cyclic polytopes 
to a bigger family of integral polytopes, “lattice-face polytopes,’ and showed that 
their Ehrhart polynomials are also in the simple form of (2.6) [56, 58]. Later in [59], 
the author improved her results by introducing a notion of “higher integrality,’ which 
we will detail below. 

Recall that a lattice point is also called an integral point. A point can be considered 
as a O-dimensional affine space. We first extend this concept of integrality to higher- 
dimensional affine spaces: An ¢-dimensional affine space W in R¢ is integral if 


n@-9 (WZ) = Z!. 


Note that this definition with £ = 0 is consistent with the original definition of an 
integral point. 


Example 2.4.2 (lines in R”). See the left side of Fig. | for examples of 1-dimensional 
affine space in R*. The black lines are integral, while the red lines are not integral. 
For the slanted red line, say L;, we have me-D(L AZ) = Z/4Z. For the vertical 
red line, say L2, we have 1?-)(L7N Z?) = Z°. 


Note that in the above example, even though L, is not integral, after the projection, 
we still get a 1-dimensional lattice, which has the same dimension as L. In this case, 
we say L, is in general position. On the contrary, L> is not in general position. 


Definition 2.4.3. Suppose 0 < k < d. A d-dimensional polytope P is k-integral if 
for any face F of P of dimension less than or equal to k, the affine hull aff(F) of F 
is integral. 

In particular, when k = d, we call P a fully integral polytope. 


Remark 2.4.4. With the above definition, lattice-face polytopes, introduced in [56, 
58], can be defined as polytopes that can be triangulated into fully integral simplices, 
which is a property any (integral) cyclic polytope has. Therefore, any cyclic polytope 
or lattice polytope is fully integral. 
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The main result in [59] is a complete description for the Ehrhart coefficients of a 
k-integral polytope in terms of volumes of projections and Ehrhart polynomials of 
slices. 


Definition 2.4.5. For any y € 1“~(P), we define the slice of P over y, denoted 
by ma_x(y, P), to be the intersection of P with the inverse image of y under 7¢~), 


Recall that [¢*] f (t) denotes the coefficient of t* of a polynomial f(t). 


Theorem 2.4.6 (L.). If P is a k-integral polytope, then 


Vol(x“~‘(P)) fO<f<k, 


[ti(P, 1) = ; 

(t(D yentoerynat mae, PO) Fk+L Sed. 
Therefore, if P is fully integral, the Ehrhart polynomial of P is in the form of (2.6), 
and thus P is Ehrhart positive. 


Because both cyclic polytopes and lattice-face polytopes are fully integral polytopes, 
the above theorem generalizes results in [56-58]. 

The following is an example showing how to use Theorem 2.4.6 to obtain the 
Ehrhart polynomial of a 1-integral polytope. 


Example 2.4.7 (Example of Theorem 2.4.6). Consider the 3-dimensional polytope 
P =conv{(0, 0, 0), (4, 0, 0), (3, 6, 0), (2, 2, 2)} CR’, 


which is illustrated on the right side of Fig. 1. One checks that P is 1-integral. Clearly 
a?) (P) = [0, 4] and r®(P) = 0. By the first part of Theorem 2.4.6, 


(t']i(P, t) = Vol, ((0,4]) =4, and [r°]i(P, t) = Volo(0) = 1. 


For the higher Ehrhart coefficients of P, we need to compute the Ehrhart polyno- 
mials of slices of P over lattice points in na?) (P) = [0, 4]. In the picture, the three 
shaded triangles are the slices of P over the lattice points 1, 2, and 3. The slices of P 
over lattice points 0 and 4 are the single points (0, 0, 0) and (4, 0, 0), respectively. 
We calculate the Ehrhart polynomials of all five slices, by summing which up we 
obtain 81° + 10¢ + 5. Then the second part of Theorem 2.4.6 says that 


(7]i(P,t) =8 and [#7]i(P,t) = 10. 


Therefore, 
i(P, t) = 80? + 1007 + 4¢ +1. 


Recall that the face poset of a polytope P is the set of all faces of P ordered 
by inclusion. We say two polytopes have the same combinatorial type if they have 
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the same face poset. As a by-product of the study of Ehrhart polynomials of full- 
integral polytopes, we can also show that Ehrhart positivity is independent from 
combinatorial types of polytopes [56]. 


Theorem 2.4.8 (L.). For any rational polytope P, there exists a polytope P' with 
the same face lattice such that P' satisfies the higher integrality condition and thus 
is Ehrhart positive. 


Sketch of proof. First, by choosing appropriate bases for our underlying lattice Z“, 
we may assume that the affine hull of any face of P is in general position. 
Next, for any s = (5),...,5¢) € Z4 and x € R®@, we define 


SX = (S]X], 82X2,..., SqXq). 


So s is an operator on R¢ that dilates points with different scalars at different coordi- 
nates. We observe that for any ¢-dimensional affine space W C R¢ that is in general 
position, there exist (positive) integer scalars c;,..., cg such that for any s € Zi95 
if CmSm divides s+, foreach m € {1,2,..., €}, then 


sx*W:={sxew:wew} 


is integral. For example, for the slanted red line L; appeared in Example 2.4.2, one 
checks that whenever s = (sj, 52) satisfies 4s, divides sz, the affine space s * L; is 
integral. Hence, we can choose c; = 4. 

Since P has finitely many faces, we can apply the above operations inductively 
on dimensions of faces to obtain a full-integral polytope P’ that actually defined as 
sx P forsomes € Zig: 


Remark 2.4.9. There are a lot of properties of polytopes people study other than 
Ehrhart positivity, such as “normality,” “integer decomposition property” (or IDP), 
and “existence of a (regular) unimodular triangulation.” For the majority of them, 
even if you start with a polytope P that does not have a certain property, dilating 
P with a large enough scalar often yields a polytope with the desired property (see, 
e.g., [19, 30, 39]). Clearly, simple dilations would not change the answer to the 
Ehrhart positivity question for any polytope. After all, i(k P, t) = i(P, kt). Hence, 
the Ehrhart coefficients of a dilation of P have exactly the same sign pattern as 
Ehrhart coefficients of P. 

However, our proof of Theorem 2.4.8 says that dilating in different directions with 
different scalars can change a non-Ehrhart-positive polytope to a Ehrhart-positive 
one. 
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3. McMullen’s Formula and Positivity of Generalized 
Permutohedra 


The main purpose of this section is to study the Ehrhart positivity conjecture for 
generalized permutohedra. After reviewing previously known results supporting this 
conjecture, we introduce McMullen’s formula, which is a formula for computing 
the number of lattice points inside polytopes. This provides us a way of attacking 
the question of Ehrhart positivity by reducing the problem to “a-positivity.” We 
then discuss the author’s joint work [24, 27] with Castillo on the Ehrhart positivity 
conjecture of generalized permutohedra using this approach. 


3.1 Motivation and Evidence 


In this part, we discuss the motivation for considering the Ehrhart positivity con- 
jecture of generalized permutohedra and prior work by Postnikov which provides 
evidence for this conjecture. We start by formally defining generalized permutohe- 
dra, the main family of polytopes we study in this section. Whenever we talk about 
generalized permutohedra, we have D = d + 1. 


3.1.1 Definition and First Positivity Conjecture 


Given a strictly increasing sequence a = (a1, Q2,..., Qd+1) € R¢+!, we define the 
usual permutohedron associated with a as 


Perm(a) := conv ((ax(1), @nQ), +++» @md+1)) 1 T € Gay) 


In particular, if a = (1,2,...,d-+ 1), we obtain the regular permutohedron [Tg 
considered in Example 2.3.2. In [82], Postnikov defined generalized permutohedra 
to be polytopes that can be obtained from usual permutohedra by moving vertices 
while preserving all edge directions. (Note that in this definition, edges are allowed 
to degenerate, and hence vertices can collapse.) 

In [31], De Loera, Haws, and Koeppe study the Ehrhart polynomials of matroid 
base polytopes, and conjecture those all have positive coefficients. However, it turns 
out that every matroid base polytope is a generalized permutohedron [1, Section 2]. 
In [24, 27], Castillo and the author generalize the conjecture of De Loera et al. to all 
integral generalized permutohedra: 


Conjecture 3.1.1 (Castillo-L.). All integral generalized permutohedra are Ehrhart 
posSitive. 
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Indeed, due to Postnikov’s work, a big family of generalized permutohedra is 
already known to be Ehrhart positive, which provides a strong evidence to the above 
conjecture. We describe his work below. 


3.1.2 Ehrhart Positivity of Type-)) Generalized Permutohedra 


In [82], Postnikov considers Minkowski sums of dilated simplices: For any nonempty 
subset J C [d + 1], define the simplex 


A; :=conv{e; : i € J}. 
Lett y=(7 : OAIC[d4+1)e (Reo) be a vector indexed by nonempty 
subsets of [d + 1] with nonnegative entries. We define the polytope 


PP(yy:= > oyrAr 
O#IC{d+1] 


as the Minkowski sum of the simplices A; dilated by the factor y;. Postnikov shows 
that py (y) is always a generalized permutohedron [82, Proposition 6.3]; however, 
not every generalized permutohedron can be expressed as py (y) for some y [82, 
Remark 6.4]. Therefore, we call py (y) a type-Y generalized permutohedron. 

Postnikov then reformulates the construction of e (y) using bipartite graphs: Let 
G be a subgraph of the bipartite graph K..a.1 without isolated vertices. Label the 
vertices of G on the left by /,, /2,..., 7. and vertices on the right by 71, 72, ..., Ta41- 
For each | < j <c, we let 


W={ieldt+]] : {l;, ri} is an edge of G}. 


For any (1, y2,---, Ve) € (Reo) , we define the polytope 
Pe, 79) = Dae 4 
j=l 


Remark 3.1.2. Itis clear that Pg(y), y2,..., Yc) is the type-Y generalized permuto- 
hedron py (y) where y; = > per ap Conversely, the type-Y generalized permu- 


tohedron py (y) is the polytope Pg(y) where G is the subgraph of K a+1_4 g44 such 
that left vertices of G are indexed by nonempty subsets J of [d + 1], and the left 
vertex /; is adjacent to the right vertex r; if and only ifi € J. 


In [82, Section 11], Postnikov defines the “trimmed generalized permutohedron” 
as the “Minkowski difference” of Pg(y1,..., Yc) and the simplex Ajg+1). By pro- 
viding a formula for the number of lattice points in a trimmed generalized permu- 
tohedron, he obtains a formula for the number of lattice points in Pg(y1,..., Ye) 
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[82, Theorem 11.3], which leads to an expression for the Ehrhart polynomial of 
PeQ1,---, Ye) as a summation over G-draconian sequences. 


Definition 3.1.3 (Definition 9.2 in [82]). A sequence of nonnegative integers g = 
(91, 92,---,Gc) iS a G-draconian sequence if ee gj; =a and for any subset 


{its +++) de} S [e], we have |17 U--- UT? > gj, +--+ +9441. 


Theorem 3.1.4 (Postnikov). Suppose G is a subgraph of K¢,a41 without isolated 
vertices such that i? = [d+ 1). Let yj,..., ye € N. Then the Ehrhart polynomial 


of Pg(1,--+, Ye) is given by 


1 Cc 
ae 8) (C9) 
k=2 


9 
where the summation is over all G-draconian sequences g = (g),.--, 9c). 


Similar to the results discussed in Sects. 2.1.3 and 2.1.4, it follows from Lemma 
2.1.1/(b) that the Ehrhart polynomial described in the above theorem has positive 
coefficients. Thus, by Remark 3.1.2, we immediately have the following: 


Corollary 3.1.5. Any integral type-Y generalized permutohedron is Ehrhart posi- 
tive. 


Note that as we pointed out above, type-YV generalized permutohedra do not 
contain all generalized permutohedra. Thus, Conjecture 3.1.1 does not follow from 
the above result. 


Example 3.1.6 (Pitman-Stanley polytopes again). Let G be a subgraph of Ky+1,4+1 
where foreach j € [d + 1], the left vertex /; is adjacent to right verticesr;,rj4i,..., 
rg41. Then for any y = (yj,..-, Ya41) € N*!, 


d+1 


Po(y) = Pe, - +s Yat) = > yj Atari 
j=l 


where [j7,d+ 1] ={j, 7 +1,...,d+ 1}. It follows from Proposition 6.3 of [82] 
that the inequality description of this polytope is 


d+l1 d+l1 


i i 
Pg(y) = ;\x eRe! . x; > 0 and bes < Yo» fori =1,2,...,d —1, and bes = bes 
j=l j=l j=l j=l 


It is easy to see the map 7 : R¢+! — R¢ that ignores the last coordinate of a point 
induces a unimodular transformation from Pg(y) to the Pitman-Stanley polytope 
PSa(y) considered in Sect. 2.1.3, where Y = (yi, y2,---, Ya) 

One can also check that the G-draconian sequences for the graph G given in this 
example are those g = (91, .--, Ja+1) € N¢*! satisfying 
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d k 
Yo 9) =4, gai =0, and )°g; > k fork =1,2,...,d—1. 


j=l j=l 
Hence, it can be verified that Theorem 2.1.2 is a special case of Theorem 3.1.4. 


The family of type-Y generalized permutohedra not only includes the Pitman- 
Stanley polytope as we have seen in the example above, but also includes associa- 
hedra, cyclohedra, and more (see [82, Section 8]). However, it follows from work by 
Ardila, Benedetti, and Doker that type-Y generalized permutohedra do not contain 
all matroid base polytopes [1, Proposition 2.3 and Example 2.6]. Therefore, Corol- 
lary 3.1.5 does not settle either Conjecture 3.1.1 or the Ehrhart positivity conjecture 
on matroid base polytopes by De Loera et al. [31]. 


3.2. McMullen’s Formula, a-Positivity, and a Reduction 
Theorem 


The goal of this part is to introduce McMullen’s formula and discuss why it is a good 
tool to show Ehrhart positivity of a family of polytopes constructed from a fixed 
projective fan when an a-construction satisfies certain valuation properties. (We 
will discuss in Sect.3.3 that generalized permutohedra form a family of polytopes 
constructed from the Braid fan. Hence, the techniques introduced here are relevant 
to our question.) 

Throughout the rest of this section, we let V be a subspace of R? and V* be the 
dual space of V. For any polytope P, we use the notation lin(P) to denote the linear 
space obtained by shifting the affine span aff(P) of P to the origin. 


3.2.1 Cones 


We need the concepts of cones, particularly feasible cones and normal cones, before 
we Start our discussion. 

A (polyhedral) cone is the set of all nonnegative linear combinations of a finite 
set of vectors. A cone is pointed if it does not contain a line. 


Definition 3.2.1. Suppose P is a polytope satisfying lin(P) C V. 
(i) The feasible cone of P at F is: 


feone(F, P) := {ue V:x+06u € P for sufficiently small 6}, 


where x is any relative interior point of F. (It can be checked that the definition 
is independent of the choice of x.) 
The pointed feasible cone of P at F is 
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fcone?(F, P) = fcone(F, P)/lin(F). 
(ii) Given any face F of P, the normal cone of P at F with respect to V is 
nconey (F, P) := {u eV* : (u, p,) => (u, po), Vp, Ee F, Var € Pi. 


Therefore, nconey(F, P) is the collection of linear functionals u in V* such 
that w attains maximum value at F over all points in P. 

The normal fan Xy(P) of P with respect to V is the collection of all normal 
cones of P. 


Normal cones and pointed feasible cones are related by polarity. 


Definition 3.2.2. Let K C V* be acone, and let W be the subspace of V* spanned 
by K. (So W* is a quotient space of V.) The polar cone of K is the cone 


K°={ye W* : (x,y) <0, Vx € K}. 


Lemma 3.2.3 (Lemma 2.4 of [24]). Suppose P is a polytope satisfying lin(P) C V 
and F is a face of P. Then (nconey (F, P))° is a pointed cone, and is invariant under 
the choice of V. So we may omit the subscript V and just write (ncone(F, P))°. 
Furthermore, 

ncone(F, P)° = fcone”(F, P). 


3.2.2 McMullen’s Formula and a Refinement of Positivity 


In 1975 Danilov asked, in the context of toric varieties, whether it is possible to 
construct a function a such that for any integral polytope P, we have 


IPNZ?|= ° a(F, P) nvol(F), (3.1) 


F:a face of P 


where a(F’, P) depends only on the normal cone of P at F, and nvol(F) is the 
volume of F normalized to the lattice aff(F) NZ”. 

McMullen [67] was the first to confirm the existence of (3.1) in a nonconstructive 
way. Hence, we refer to the above formula as McMullen’s formula. Pommersheim 
and Thomas [81] provide a canonical construction based on choices of flags. Berline 
and Vergne [11] give a construction in a computable way. Most recently, Ring and 
Schiirmann [87] give another construction for McMullen’s formula based on a choice 
of fundamental cells. 

Before discussing a specific construction, even the existence of McMullen’s for- 
mula has interesting consequences. In fact, it was one of the ingredients used in 
proving the results on higher integrality conditions discussed in Sect.2.4.2. More 
importantly, it provides another proof for Ehrhart’s theorem (Theorem 1.1) as well 
as a refinement of Ehrhart positivity. Note that pointed feasible cones do not change 
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Fig. 2 Different a-constructions 


when we dilate a polytope. Thus, applying McMullen’s formula to ¢ P and rearrang- 
ing coefficients, we obtain a formula for the function i(P, f) : 


dim P 
i(P,t)= YS ( 2 a(F, Prva) re i 


k=0 \F:k-dimensional face of P 


Hence, i(P,t) is a polynomial in t of degree dim P, and the coefficient of t* in 
i(P, t) is given by 


[t]i(P, t) = > a(F, P)nvol(F). (3.2) 


F:k-dimensional face of P 


Example 3.2.4. Setting & = 0 in (3.2), we obtain 


[1i(P,t) = 2 a(v, P)nvol(v). 


vu:vertex of P 


Note that [¢°]i(P, ft) is the constant term of the Ehrhart polynomial of P, which is 
known to be | for any integral polytope P. Furthermore, the normalized volume of 
any vertex is 1. Hence, the above equation becomes 


y a(v, P) = 1. 


vu:vertex of P 


See Fig. 2 for a-values of the vertices of the triangle P = conv((0, 0), (2, 0), (2, 1)) 
arising from different constructions. 


Since nvol(F’) is a positive number, it follows from (3.2) that a-values refine 
Ehrhart coefficients. We say a polytope P is a-positive for k-faces if a(F, P) is 
positive for all k-dimensional faces F of P and say P is a-positive if all a’s associated 
to P are positive. The following result immediately follows from expression (3.2). 


Lemma 3.2.5. Suppose a is a solution to McMullen’s formula. Let P be an integral 
polytope. For a fixed k, if P is a-positive for k-faces, then the coefficient of t* in the 
Ehrhart polynomial i(P, t) of P is positive. 

Hence, if P is a-positive, then P is Ehrhart positive. 
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3.2.3. BV-Construction and the Reduction Theorem 


At the first glance, a-positivity, being a refinement of Ehrhart positivity, is a more 
difficult question to consider. However, for a-constructions that satisfy certain prop- 
erties, studying a-positivity instead does not necessarily make the problem harder. 
Berline and Vergne [11] give such an a-construction, of which we give a quick review 
below. Recall that the indicator function of aset A C V is the function[A]: V > R 
defined as [A](x) = 1 if x € A, and [A](x) = Oif x ¢ A. The algebra of rational 
cones, denoted by C(V), is the vector space over Q spanned by the indicator func- 
tions [C] of all rational cones C C V. We consider C(V) a subspace of the vector 
space of all functions on V. Hence, in general, the indicators [C] of rational cones 
do not form a basis of C(V) since there are many relations among them. 


Theorem 3.2.6 (Berline-Vergne). There exists a function V from the set of indicator 
functions [C] of rational cones C in V to R with the following properties: 


(P1) W induces a valuation on the algebra of rational cones in V, i.e., V induces a 
linear transformation from C(V) to R. 

(P2) Ifacone C contains a line, then V([C]) = 0. 

(P3) W is invariant under orthogonal unimodular transformation, thus, is symmet- 
ric about coordinates, that is, invariant under rearranging coordinates with 
SIQNS. 

(P4) Setting 

a(F, P) := W([fcone’ (F, P)]), (3.3) 


gives a solution to McMullen’s formula. 


We refer to Berline-Vergne construction of WY and a as BV-construction and BV-a- 
valuation, respectively. If a is the BV-a-valuation, we use the terminology BV-a 
-positivity instead of a-positivity. 

Properties (P1) and (P2) are the “certain valuation properties” we mentioned at 
the beginning of Sect. 3.2. The following Reduction Theorem lays out a consequence 
of these two properties. 


Theorem 3.2.7 (Castillo-L., Reduction Theorem [24]). Suppose W is a function 
from the set of indicator functions of rational cones C in V to R such that properties 
(P1) and (P2) hold, and suppose « is defined as in (3.3). 

Let P and Q be two polytopes such that lin(P) and lin(Q) are both subspaces 
of V. Assume the normal fan Xy(P) of P with respect to V is a refinement of the 
normal fan Xy(Q) of Q with respect to V. 

Then for any fixed k, if P is a-positive for k-faces, then Q is a-positive for k-faces. 


One important implication of the Reduction Theorem is that we can reduce the 
problem of a-positivity of a family of polytopes constructed from a fan to the problem 
of a-positivity of a single polytope in the family. 
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Definition 3.2.8. Let © be a projective fan in V%, i.e., a fan that is a normal fan of 
some polytope. Let Poly(2) be the set of polytopes Q whose normal fan Xy(Q) 
with respect to V coarsens &. 


Corollary 3.2.9. Assume the hypothesis on VY and a in Theorem 3.2.7. Let X& bea 
projective fan in V*, and let P be a polytope such that Xy(P) = X. Then a-positivity 
(for k-faces) of P implies a-positivity (for k-faces) of Q for any Q € Poly(X). 

Assume further that a is a solution to McMullen’s formula. Then for any integral 
polytope Q € Poly(), a-positivity for k-faces of P implies the coefficient of t* in 
i(Q, t) is positive. Hence, a-positivity of P implies Ehrhart positivity of Q. 


Proof. The first part follows directly from the Reduction Theorem, and the second 
assertion follows from the first part and Lemma 3.2.5. 


Therefore, even though proving a-positivity is more difficult than proving Ehrhart 
positivity for an individual polytope, it could be easier if we consider a family of 
polytopes Poly(X) associated to a fixed projective fan 4, as we only need to prove 
a-positivity for one polytope in the family. Finally, because the BV-construction 
satisfies properties (P1), (P2), and (P4), all the results discussed above apply to the 
BV-construction or the BV-a-valuation. These ideas are illustrated by Example 3.3.3 
below. 


3.3 Positivity of Generalized Permutohedra 


In this part, we apply the Reduction Theorem to reduce our first conjecture— 
Conjecture 3.1.1—to a conjecture on a-positivity of regular permutohedra. Then 
we report partial progress made on both conjectures by using McMullen’s formula 
with BV-q-valuation [24, 27]. 


3.3.1 Second Positivity Conjecture 


Postnikov, Reiner, and Williams give several equivalent definitions for generalized 
permutohedra, one of which uses concepts of normal fans [83, Proposition 3.2]. 
Recall that the Braid fan, denoted by Brg, is the complete fan in R¢*! given by the 
hyperplanes x; — x; = 0 for alli 4 j. 


Proposition 3.3.1 (Postnikov-Reiner-Williams). A polytope P in V = R@*! is a 
generalized permutohedron if and only if its normal fan Uy (P) with respect to V is 
refined by the Braid fan Brg. 


Using the notation we give in Definition 3.2.8, the above result precisely says that 
the family of generalized permutohedra in R¢+! is Poly(Br,). Furthermore, it follows 
from [82, Proposition 2.6] that any usual permutohedron in R¢*+! has the Braid fan 
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Q1 


Fig. 3. Examples for Corollary 3.2.9 


Brg as its normal fan. In particular, the normal fan of the regular permutohedron IT 
is Brg . In [24], Castillo and the author use these results together with the Reduction 
Theorem and its consequence (i.e., Corollary 3.2.9) to reduce Conjecture 3.1.1 to the 
following conjecture: 


Conjecture 3.3.2 (Castillo-L.). Every regular permutohedron Yq is BV-a-positive. 


The following example demonstrates how Corollary 3.2.9 works and why 
Conjecture 3.1.1 can be reduced to Conjecture 3.3.2. 


Example 3.3.3. Let P, Q1, Q2, and Q3 be the 2-dimensional polytopes together 
with their normal fans shown in Fig.3. One notices that P is the regular permuto- 
hedron IT, whose normal fan is Br2, and each Q; is a generalized permutohedron 
whose normal fan coarsens Brp. 

All the BV-a-values of the six vertices of P are 1/6. Since Q, has the same normal 
fan as P, all of its six vertices also have the same BV-a-values. Now the normal fan 
of Q> coarsens that of P. In particular, if we let v be the vertex on the bottom-left 
of Q», then the normal cone ncone(v, Q2) of Q> at v is the union of the normal 
cones of P at two of its vertices. It is a consequence of the “valuation properties” 
(P1) and (P2) that the BV-a-values a(v, Q2) is the sum of the BV-a-values of these 
two vertices of P. Therefore, as shown in the figure, a(v, Q2) = 1/6 + 1/6 = 1/3. 
One sees that similar phenomenon happens for the polytope Q3. 

The above discussion shows that even if we did not know the BV-a-values of 
vertices of P, because each BV-a-value arising from Q; is a summation of a subset 
of BV-a-values of vertices of P, BV-a-positivity of vertices of the regular per- 
mutohedron P = II, would imply BV-a-positivity of vertices of the generalized 
permutohedron Q; and thus would imply the constant Ehrhart coefficient of Q; is 
positive. 


Conjecture 3.3.2 was the main conjecture studied in [24], and partial progress was 
made on proving it, which gave us corresponding partial results on Conjecture 3.1.1. 


3.3.2 Partial Results 


The first approach of attacking Conjecture 3.3.2 is to directly compute BV-a- 
valuations. In order to do that, we need to compute the BV-construction VY. One 
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obvious benefit of considering Conjecture 3.3.2 instead of Conjecture 3.1.1 is that 
in each dimension there is only one regular permutohedron, and thus there are a 
limited number of BV-a-values or Y-values to be computed, especially for small d. 
Therefore, by explicit computation, we obtain the following result. 


Theorem 3.3.4 (Castillo-L.). Ford < 6, the regular permutohedron Vg is BV-a- 
positive. Therefore, all the integral generalized permutohedra (including matroid 
base polytopes) of dimension at most 6 are Ehrhart positive. 


Next, instead of focusing on all the coefficients of Ehrhart polynomials, we study 
certain special coefficients. Note that the first, second, and last Ehrhart coefficients 
are always positive, so we only consider other Ehrhart coefficients. Correspondingly, 
we need to know how to compute the BV-construction VY (C) of cones C of dimension 
2,3,...,d — 1. The computation of the function W is carried out recursively. Hence, 
it is quicker to compute WV for lower-dimensional cones. As a result, the value of 
a(F, P) is easier to compute if F is a higher-dimensional face. 

In general, the computation of Y(C) is quite complicated. However, when C is a 
unimodular cone, computations are greatly simplified. In dimensions 2 and 3, with 
the help of Maple code provided by Berline and Vergne, simple closed expression 
for Y of unimodular cones can be obtained [24, Lemmas 3.9 and 3.10]. Applying 
these formulas to zy, we obtain the second partial result toward Conjectures 3.3.2 
and 3.1.1: 


Theorem 3.3.5 (Castillo-L.). For any d, and any face F of Wg of codimension 2 or 
3, we have a(F, Wg) is positive, where a is the BV-a-valuation. 

Hence, the third and fourth Ehrhart coefficients of any integral generalized per- 
mutohedron (including matroid base polytopes) are positive. 


Finally, the last partial result presented in [24] is the following: 


Lemma 3.3.6 (Castillo-L.). For any d < 500, and any edge E of a, we have 
a(E, Tq) is positive, where a is the BV-a-valuation. 

Hence, the linear Ehrhart coefficient of any integral generalized permutohedron 
(including matroid base polytopes) of dimension at most 500 is positive. 


As we have discussed above, in order to compute the B V-a-values for an edge of a 
d-dimensional polytope, we have to compute the W-value of a (d — 1)-dimensional 
cone, which is extremely difficult for large d if we use Berline-Vergne algorithm 
directly. Therefore, we use a completely different strategy. Recall Property (P3) of 
the BV-construction, which says that YW is symmetric about coordinates. Note that 
the regular permutohedron IT, is a polytope with much symmetry. So a lot of BV-a- 
values of IIq coincide. In particular, we can separate edges of IT, into to [4] groups, 
where edges in each group share the same BV-a-values. 


(Idea of Proof for Lemma 3.3.6). If we know the a-values for a give polytope P, 
Eq. (3.2) gives us a way to compute the Ehrhart coefficients. However, we can also 
use (3.2) in the other direction: Suppose we know the linear coefficient of i(P, t), 
Eq. (3.2) gives us an equation for a-values arising from edges of P : 
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ss a(E, P)nvol(E) = [t']i(P, 2). 


E:edge of P 


The a-values for the regular permutohedron also appear in other generalized 
permutohedra as all of them are in the family Poly(Br,). Therefore, if we can find 
[4] “independent” generalized permutohedra for which we know their linear Ehrhart 
coefficients, then we can set up a [$] x [4] linear system for the [¢] a-values 
arising from edges of I1z. Solving the system, we obtain all these a-values. See [24, 
Example 3.15] for an example of how we can solve a linear system to find a’s. 

Recall that Postnikov gives explicit formulas for the Ehrhart polynomials of type- 
Y generalized permutohedra (see Theorem 3.1.4). Among all the nontrivial Ehrhart 
coefficients, the linear terms can be easily described. Using these, we were able 
to set up, for each d, a desired linear system which is actually triangular. Solving 
the system for d < 500, we confirmed positivity of all [$] a’s arising from edges 
of T1q. 
Equivalence Statements. In addition to the partial results discussed above, two 
equivalent statements to Conjecture 3.3.2 were discovered. The first states that Con- 
jecture 3.3.2 holds if and only if the mixed lattice point valuation on hypersimplices 
is positive [24, Corollary 5.6]. 

The second equivalent statement is in terms of Todd classes. The BV-construction 
gives one way to write the Todd class of the permutohedral variety in terms of the 
toric invariant cycles. We can show that if there is any way of writing such class as 
a positive combination of such cycles, then the BV-a-valuation is one of them. (See 
[25, Proposition 7.2] or [23].) 


4 Negative Results 


In this section, we will discuss examples and constructions of polytopes with neg- 
ative Ehrhart coefficients. We start in Sect.4.1 with the well-known Reeve tetrahe- 
dra, a family of 3-dimensional polytopes with negative linear Ehrhart coefficients. 
Constructions given in Sect.4.2 were motivated by a refinement of Question 1.2, 
considering all possible sign patterns of Ehrhart coefficients. Examples studied in 
Sects.4.3 and 4.4 provide negative answers to Question 1.2 for different families 
of polytopes (such as smooth polytopes and order polytopes), which will be sum- 
marized in Sect. 4.5. Finally in Sect.4.6, we give negative examples addressing the 
question of whether Minkowski summation preserves Ehrhart positivity. 

As mentioned before, due to the fact that the first, second, and last Ehrhart coef- 
ficients are always positive, given a d-dimensional polytope P, we need to ask the 
positivity question only for the coefficients of t2-2, 4-3 st in i(P,t). We call 
these coefficients the middle Ehrhart coefficients of P. 
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4.1 Reeve Tetrahedra 


For d < 2, there are no middle Ehrhart coefficients. Hence, possible examples with 
negative Ehrhart coefficients can appear only in dimension 3 or higher. The first 
example comes in dimension 3 : The Reeve tetrahedron T,, is the polytope with 
vertices (0, 0, 0), (1, 0, 0), (0, 1, 0), and (1, 1, m), where m is a positive integer. Its 
Ehrhart polynomial is 


12 — 


‘ mM 3 2 m 
i(Tm.t) = ¢t +04 t+. 


One sees that the linear coefficient is 0 when m = 12 and is negative when m > 13. 


4.2 Possible Sign Patterns 


Motivated by the example of Reeve tetrahedra, Hibi, Higashitani, Tsuchiya, and 
Yoshida study possible sign patterns of middle Ehrhart coefficients and ask the fol- 
lowing question: 


Question 4.2.1 (Question 3.1 of [47]). Given a positive integer d > 3 and integers 
1 <i, <--- <i, <d—2, does there exist a d-dimensional integral polytope P 
such that the coefficients of t",...,¢'7 of i(P, ft) are negative, and the remaining 
coefficients are positive? 


The following is the main result in [47] providing a partial answer to 
Question 4.2.1. 


Theorem 4.2.2 (Hibi-Higashitani-Tsuchiya- Yoshida). Let d > 3. The following 
Statements are true. 


(a) There exists an integral polytope P of dimension d such that all of its middle 
Ehrhart coefficients are negative. 

(b) For each 1 <k <d-—2, there exists an integral polytope P of dimension d 
such that [t*]i(P, t) is negative and all the remaining Ehrhart coefficients are 
positive. 


The proof of both parts of the theorem is by construction. We will briefly discuss 

the construction for Theorem 4.2.2/(a) and refer interested readers to the original 
paper [47] for the other construction. 
Sketch of proof for Theorem 4.2.2/(a). Let L, := [0,], which is a 1-dimensional 
polytope, and its Ehrhart polynomial is i(L,, t) = nt + 1. Define the polytope P to 
be the direct product of (d — 3) copies of Lg_3 and one copy of the Reeve tetrahedron 
Tm. Then P is a d-dimensional polytope with Ehrhart polynomial 


p(ae.2) =i(La-3. 9 in = (d= 30-4 DSF +? 4 t+ i). 
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Fig. 4 From 3()2 to 
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at distance 1 


The coefficients of the Ehrhart polynomial of P can be explicitly described, from 
which one can show that all middle Ehrhart coefficients are negative for sufficiently 
large m. 

In addition to Theorem 4.2.2, Hibi et al. also show that answer to Question 4.2.1 
is affirmative for d < 6 [47, Proposition 3.2]. Note that for d < 6, there are at most 
3 middle Ehrhart coefficients. Later, Tsuchiya (private communication) improved 
their result showing that any sign pattern with at most 3 negatives is possible for the 
middle Ehrhart coefficients. Unfortunately, it is not currently clear how to extend the 
techniques used to prove this result to attack the question of whether any sign pattern 
with 4 negatives can occur. So Question 4.2.1 is still wide open. 


4.3 Smooth Polytopes 


A d-dimensional integral polytope P is called smooth (or Delzant) if each vertex is 
contained in precisely d edges, and the primitive edge directions form a lattice basis 
of Z“. In [18, Question 7.1], Bruns asked whether all smooth integral polytopes are 
Ehrhart positive. In [26], Castillo, Nill, Paffenholz, and the author show the answer 
is false by presenting counterexamples in dimensions 3 and higher. The main ideas 
we used was chiseling cubes and searching for negative BV-a-values. 

The first set of examples we construct is as follows: For positive integers a > 2b, 
we let Qy(a, b) be the polytope obtained by chiseling all vertices of aLlg at distance 
b. (See Fig. 4 for an example.) Using inclusion-exclusion and the fact that the BV-a- 
values of cubes and standard simplices can be obtained easily due to property (P3), 
we obtain explicit formulas for all BV-a-values arising from Py(a, b), which we 
use to search for negative BV-a-values. The first negative values appear at d = 7, 
suggesting that we might have a negative Ehrhart coefficient in Q7(a, b). By direct 
computation, we are able to show that for some choices of (a, b), e.g., (5, 2), the 
polytope Q,(a, b) has a negative linear Ehrhart coefficient for any d > 7. Therefore, 
we have the following result [26, Proposition 1.3]: 


Proposition 4.3.1 (Castillo-L.-Nill-Paffenholz). Let Ny be the normal fan of 
Qa(a, b). For d < 6, any d-dimensional smooth integral polytope with normal fan 
Nq is Ehrhart positive. For each d > 7, there exists a d-dimensional smooth integral 
polytope with normal fan Ny whose linear Ehrhart coefficient is negative. 
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Remark 4.3.2. The polytope Qa(a, b) is not only a smooth polytope, but also a 
“type-B generalized permutohedron.” The generalized permutohedra considered in 
Sect.3 are of type A as the corresponding normal fan, Bry, is constructed from 
the type A root system. As a consequence, a polytope P is a (type-A) generalized 
permutohedron if and only if each edge direction of P is of the form of e; — e; for 
some i # j. Similarly, we can define a polytope P in R@ is a type-B generalized 
permutohedron if each edge direction of P is in the form of e; + e; for some i ¢ j 
or of the form +e; for some /. It is then straightforward to verify that Qa(a, b) isa 
type-B generalized permutohedron. 


Using the idea of iterated chiseling cubes, we then improve the dimension range 
of our counterexamples from d > 7 to d > 3. (See [26, Section 2] for details.) 


Theorem 4.3.3 (Castillo-L.-Nill-Paffenholz). For each d > 3, there exists a 
d-dimensional smooth integral polytope P such that all of its middle Ehrhart coef- 
ficients are negative. 


Note that the above theorem is a stronger version than part (a) of Theorem 4.2.2. 
Even though the original purpose of the paper [26] was to answer Bruns’ question, 
in the process of searching for a counterexample, we obtained a separate result 
answering a different question. For positive integers a > b, we let Pa(a, b) be the 
polytope obtained by chiseling one vertex off aLlg at distance b. It is clear that 
Pa(a, b) share the same BV-a-values with Qa(a, b). Hence, it has negative BV-a- 
values at d > 7. However, it turns out any d-dimensional integral polytope P that 
has the same normal fan as Py (a, b) is Ehrhart positive [26, Lemma 3.9]. 


Corollary 4.3.4 (Castillo-L.-Nill-Paffenholz). For eachd > 7, there exists a smooth 
projective fan &, such that its associated BV-a-values contains negative values, but 
any smooth integral polytope in Poly(%) is Ehrhart positive. 

Therefore, BV-a-positivity is strictly stronger than Ehrhart positivity. 


Finally, we studied a weaker version of Brun’s question by requiring the smooth 
polytopes to be reflexive. More precisely, we asked whether all smooth reflexive poly- 
topes have positive Ehrhart coefficients. Unfortunately, the answer to this question 
is still negative. 

In fixed dimension d, there are only finitely many reflexive polytopes up to uni- 
modular transformations. Because of their correspondence to toric Fano manifolds, 
smooth reflexive polytopes were completely classified up to dimension 9 [62, 76]. 
We used polymake [2] to verify that up to dimension 8 all of them are Ehrhart 
positive, but in dimension 9 the following counterexample came up [26]: 
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Example 4.3.5. Let P be the polytope in R° defined by 


100000000 
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stat at Sho D. OG. asi d 


0 0 0 0 -1-1-1-1-4 


Using polymake [2], one can check that this polytope is smooth and reflexive with 
Ehrhart polynomial 


i(P,t) = 12477727/181441? + 12477727 /403218 + 9074291 /1512t7 + 630095/96r° 


+ 19058687/43201> + 117857/64¢4 + 3838711/9072r7 + 11915/100827 
— 6673/6301 + 1, 


which has a negative linear coefficient. 


4.4 Stanley’s Example 
In answering an Ehrhart positivity question posted on mathoverflow, Stanley gave 
the following example [100]: 


Example 4.4.1. Let Q; be the poset with one minimal element covered by k other 
elements. The Ehrhart polynomial of the order polytope O(Q,) is 


t+ 
(OQ), t) = oi. 
i=l 
One can compute that 


[t!]i(O(Qo0), t) = —168011/330 < 0. 


Hence, the linear Ehrhart coefficient of O(Q,) is negative when k = 20. 


Based on Stanley’s example, the author and Tsuchiya studied the Ehrhart posi- 
tivity question on polytopes O(Q,) for every k and gave a complete description of 


On Positivity of Ehrhart Polynomials 225 


Fig. 5 Q, and its 
corresponding planar 1 
graph G 


which Ehrhart coefficients of O(Q;) are negative [61]. The following theorem is an 
immediate consequence to this description. 


Theorem 4.4.2 (L.-Tsuchiya). The order polytope O(Q;) (defined in Example 
4.4.1) is Ehrhart positive if and only if k < 19. 


Stanley’s example and its extension are very interesting as O(Q;) belongs to a 
lot of different families of polytopes. First of all, it is an order polytope and thus is 
a (0, 1)-polytope. 

Recall that a Gorenstein polytope of codegree s is an integral polytope such that s P 
is reflexive. It follows from a result by Hibi [42] that an order polytope is Gorenstein 
if and only if the underlying poset is pure; i.e., all maximal chains have the same 
length. Clearly, Q; is pure. Thus, O(Q;) is a Gorenstein polytope. 

Finally, Mészaros, Morales, and Striker proved a result observed by Postnikov 
establishing a connection between flow polytopes of planar graphs and order poly- 
topes [72, Theorem 3.8]. Using this connection, Morales (private communication) 
observes that the order polytope O(Q,) is unimodularly equivalent to the flow poly- 
tope Fg(,0,...,0,—1), where G is the black graph on k + 1 vertices in Fig. 5. 
(Note the red part of the figure is Ox.) 


4.5. Non-Ehrhart-Positive Families 


For each of the families listed below, it is not true that all the integral polytopes in 
the family are Ehrhart positive. 


(i) Smooth polytopes. (vi) Flow polytopes. 
(ii) Type- B generalized permutohedra. 
(iii) (0,1)-polytopes. 
(iv) Order polytopes. 

(v) Chain polytopes. (ix) Smooth reflexive polytopes. 


(vii) Gorenstein polytopes. 


(viii) Reflexive polytopes. 


Furthermore, non-Ehrhart-positive examples were constructed for family (i) for 
each dimension d > 3, for family (11) for each dimension d > 7, and for families 
(iii), (iv), (v), (vi), (vii), and (viii) for each dimension d > 21. 
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Proof. The conclusion for (i) and (ii) follows from Theorem 4.3.3, Proposition 4.3.1, 
and Remark 4.3.2. Notice that the order polytope O(Q;) considered in Sect. 4.4 has 
dimension k + 1. Then the conclusion for (iii), (iv), (vi), and (vii) follows directly 
from discussion in Sect.4.4. Next, (v) follows from (iv) and Remark 2.2.6, and 
(viii) follows from (vii), the connection between Gorenstein polytopes and reflexive 
polytopes and the fact that Ehrhart positivity is invariant under dilating operations. 
Finally, (ix) follows from Example 4.3.5. 


4.6 Minkowski Sums 


Recall that in Sect. 3.1.2, we learned that the type- generalized permutohedra which 
are defined to be Minkowski sums of dilated standard simplices are Ehrhart positive. 
Noticing that standard simplices are Ehrhart positive (see Sect. 2.1.2), we asked the 
following question in the first version of this survey: 


Ts it true that if two integral polytopes P and Q are Ehrhart positive, then their 
Minkowski sum P + Q is Ehrhart positive? 


Tsuchiya (private communication) constructed afew examples, which gave a negative 
answer to the above question, shortly after it was posted. Below are two of his 
examples. 


Example 4.6.1. Let P be the 3-dimensional simplex with vertices 
(0, 0, 0, 0), (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), 
and Q the 1-dimensional polytope with vertices 
(0, 0,0, 0), (1, 19, 19, 20). 


It is easy to see that both P and Q are Ehrhart positive. However, one can check that 
P + Q is a 4-dimensional polytope with Ehrhart polynomial 


i(P + Q,t) = 10/3t* + 7/6r? — 1/3? + 17/6t + 1, 


which has a negative quadratic coefficient. 


Example 4.6.2. Let P be the 5-dimensional simplex with vertices 


(0, 0, 0, 0, 0), (1, 0, 0, 0, 0), (0, 1, 5, 15, 16), (0, 0, 1, 0, 0), (0, 0, 0, 1, 0), (0, 0, 0, 0, 1), 
and Q the 5-dimensional simplex with vertices 


(0, 0, 0, 0, 0), (1, 0, 0, 0, 0), CL, 1, 15, 15, 16), (0, 0, 1, 0, 0), (0, 1, 0, 0, 0), (0, 0, 0, 0, 1). 
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Since P is unimodularly equivalent to the standard 5-simplex, it is Ehrhart positive. 
Moreover, the Ehrhart polynomial of Q is 


i(Q, t) = 1/817 + 5/124 + 17/2423 + 19/120? + 13/6t +1, 


which also has positive coefficients. 
However, P + Q is a 5-dimensional polytope with Ehrhart polynomial 


i(P + Q, t) = 3007/402° + 359/24¢4 — 255/240? + 193/241? + 89/20t + 1, 


which has a negative coefficient. 


Notice that the polytopes given in Example 4.6.1 satisfy dim(P) + dim(Q) = 
dim(P + Q), and those in Example 4.6.2 satisfy dim(P) = dim(Q) = dim(P + 
Q). These are the two extreme situations in terms of dimensions. Therefore, even 
with some restrictions on the dimensions of P, Q, and P + Q, the answer to the 
question above is false. 


5 Further Discussion 


5.1 Ehrhart Positivity Conjectures 


We list several families of polytopes that are conjectured to be Ehrhart positive. 


5.1.1 Base-r Simplices 


Recall the definition of Aq.) given in Sect.2.2.2. For any positive integer r € P, 
we let q, = (r — 1, (r — Dr, (r — Vr’, ..., (7 — 1)r4~!) € P“ and then define the 
base-r d-simplex to be 


Bod) = Aagq,)- 


Note that if r = 1, we obtain a polytope that is unimodularly equivalent to the 
standard d-simplex. The family of base-r d-simplices are introduced by Solus in 
his study of simplices for numeral systems [91], in which he shows that the h*- 
polynomial of By,a) is real-rooted and thus is unimodal. Based on computational 
evidence, Solus makes the following conjecture [91, Section 5]: 


Conjecture 5.1.1 (Solus). The base-r d-simplex is Ehrhart positive. 


We remark that this family of A(;,,) is very different from the ones constructed 
by Payne discussed in Example 2.2.5. If r > 1, the base-r simplex By,a) always 
contains the origin as an interior point, and it follows from (1.2) that the degree of 
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h*-polynomial of By,,~) is d. Since By,,a) is not reflexive, by Corollary 2.2.9 the roots 
of its h*-polynomial are not all on the unit circle in the complex plane. Therefore, 
the techniques used to prove Ehrhart positivity for Payne’s construction would not 
work here. 


5.1.2 Birkhoff Polytopes 


The Birkhoff polytope B,, is the convex polytope of n x n doubly stochastic matrices; 
that is, the set of real nonnegative matrices with all row and column sums equal to 
one. Equivalently, B, can also be defined as the convex hull of all n x n permuta- 
tion matrices. (See [106, Chaps.5 and 6] for a detailed introduction to B,,.) There 
has been a lot of research on computing the volumes and Ehrhart polynomials of 
Birkhoff polytopes [8, 21, 32, 77]. The following conjecture was made by Stanley ina 
talk [97]: 


Conjecture 5.1.2 (Stanley). Birkhoff polytopes are Ehrhart positive. 


By checking the available data [8], the first nine i(B,, t) have the property that all 
the roots have negative real parts. More importantly, Fig.6 in [9] suggests that the 
roots of i(B,, t) form a certain pattern. Hence, it could be promising to use Lemma 
2.2.1 to attack this conjecture. 

We also remark that B, is a Gorenstein polytope (up to lattice translation) of 
codegree n. However, with aforementioned data, one can see that B,, is not h*-unit- 
circle-rooted. Hence, we cannot apply Theorem 2.2.2 to show that all roots of i(B,,, t) 
have negative real parts. 


5.1.3 Tesler Polytopes 


For any n x n upper triangular matrix M = (m;,;), the kth hook sum of M is the 
sum of all the elements on the kth row minus the sum of all the elements on the kth 
column excluding the diagonal entry: 


(mee + Me ke Ht Men) — (Mk + M2,p + FM 1,k)- 


For each a = (a,...,d,) € N", Mészaros, Morales, and Rhoades [71] define the 
Tesler polytope, denoted by Tes,(a), to be the set of all n x n upper triangular 
matrices M with nonnegative entries and of hook sum a; i.e., the kth hook sum of 
M is ax. The lattice points in Tes,(a@) are exactly Tesler matrices of hook sum a. 
When a = (1, 1,..., 1), these are important objects in Haglund’s work on diagonal 
harmonics [40]. Therefore, we call Tes, (1, 1,..., 1) the Tesler matrix polytope as 
Tesler matrices of hook sum (1, 1,..., 1) were the original Tesler matrices defined 
by Haglund. 

Another interesting example of a Tesler polytope is Tes, (1, 0, ..., 0), which turns 
out to be the Chan-Robbins- Yuen polytope or CRY polytope, a face of the Birkhoff 
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polytope. The CRY polytope, denoted by CRY, is the convex hull of all then x n 
permutation matrices M = (m;,;) such that m;,; = 0 if i > j + 2; Le., all entries 
below the subdiagonal are zeros. It was initially introduced by Chan, Robbins, and 
Yuen in [28], in which they made an intriguing conjecture on a formula for the volume 
of CRY, as a product of Catalan numbers. It was since proved by Zeilberger [107], 
Baldoni-Vergne [3], and Mészaros [68, 69]. 

Using the Maple code provided by Baldoni, Beck, Cochet, and Vergne [4], Morales 
computed the Ehrhart polynomials of both CRY polytopes and Tesler matrix poly- 
topes for small n and made the following conjecture [73]. 


Conjecture 5.1.3 (Morales). For each positive integer n, the CRY polytopeCRY,, = 
Tes, (1, 0,..., 0) and the Tesler matrix polytope Tes, (1, 1,..., 1) are both Ehrhart 
positive. 


Connection to flow polytopes. Mészaros et al. show in [71, Lemma 1.2] that for 
any a € N”, the Tesler polytope Tes, (a) is unimodularly equivalent to the flow 
polytope Fx,,,(a@), where K,,,, is the complete graph on [n + 1] and a is defined 
as in (2.2). Therefore, Tesler polytopes are flow polytopes associated to complete 
graphs. Note that the complete graph does not satisfy the hypothesis of Corollary 
2.1.5. So Conjecture 5.1.3 does not follow. 


5.1.4 Stretched Littlewood-Richardson Coefficients 


The Schur functions s, form a basis for the ring of symmetric functions. (See [96, 
Chap.7] for background on symmetric functions.) Therefore, the product of two 
Schur functions s) and s,, can be uniquely expressed as 


Vv 
SV Sy = ) Cy Sv: 


v:|v|=|A|+| 1] 


We call the coefficients cy P in the above expression the Littlewood-Richardson coef- 
ficients or LR coefficients. There are many different ways of computing Cou For 
example, it counts the number of semistandard Young tableaux T of shape v/A with 
content jz such that the reading word of T satisfies the “Yamanouchi word condition” 
[65]. One immediate consequence of these descriptions is that the LR coefficients are 
nonnegative integers. In this chapter, we use the hive model [20, 53, 54] to describe 
the LR coefficients. 

A hive of size n is a triangular array of numbers a;,; with 0 <i, j,i tj <n 
arranged on a triangular grid consisting of n* small equilateral triangles. See the left 
side of Fig.6 for how a hive of size 4 should look like. For any adjacent triangles 
{a, b, c} and {b, c, d} in the hive, they form arhombus {a, b, c, d}. The hive condition 
for this rhombus is 

b+c>a+d. (HC) 
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Fig. 6 A hive of size 4 


Suppose |v| = |A| + || with 1(v), L(A), Gu) <n. A Littlewood-Richardson-hive or 
LR-hive of type (v, 4, 1) isahive {a;,; ¢ N : 0 <i, j,i + j < n} with nonnegative 
integer entries satisfying the hive condition (HC) for all of its rhombi, with border 
entries determined by partitions v, A, 44 in the following way: ao,o = 0 and for each 
fHAZ2 haan; 


4j0-4j-10=Vj, 40,j — 40,j-1 = Aj» Ajn—j — Aj-1n—j+1i = Lk: 


With this definition, the LR coefficient cy ,, counts the number of LR-hives of type 
(v, A, p). (Note that this is independent from n as long as /(v), /(A), [(41) <n.) 

For example, if v = (4,3, 1), AX = (3,2) and uw = (2, 1), then the border of a 
corresponding LR-hive of size 4 is shown on the right side of Fig. 6. In fact, the hive 
condition will force az; = 8 and a;,. = 7. So it will be reduced to a hive of size 
3. Finally, it follows from the hive condition that 6 < a;,; < 7. Thus, we have two 
LR-hives of this type, and we conclude that Cs = 2. 

From the above description, it is not hard to see that cy , counts the number of 
lattice points inside a polytope Py P determined by the border condition and the hive 


condition. Furthermore, for any positive integer ¢, the LR coefficient c!¥ 


t,t counts 


the number of lattice points inside the r’” dilation of PY a 
tv _ Vv D 
Cty = |tPy , AZ if 


King, Tollu, and Toumazet studied c Xt i which they call the stretched Littlewood- 
Richardson coefficients and made the following conjecture [51, Conjecture 3.1]: 


Conjecture 5.1.4 (King-Tollu-Toumazet). For all partitions X, 1, v such that cy |, > 
0, there exists a polynomial f (t) = ree. int such that f (0) = land f(t) = Chi 
for all positive integers t. 

Furthermore, all the coefficients of f (t) are positive. 


One notices that if Py , iS an integral polytope, then the polynomiality part of 
the above conjecture follows from Ehrhart’s theorem. However, in general, Py i is 
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a rational polytope, which only implies that oo. i, 18 a quasi-polynomial with some 
period. Nevertheless, the assertion of polynomiality in the above conjecture was 
established first by Derksen and Weyman [33] using semi-invariants of quivers, and 
then by Rassart [86] using Steinberg’s formula [104] and hyperplane arrangements. 
Hence, the polynomial asserted in Conjecture 5.1.4 can be considered to be an Ehrhart 
polynomial, and positivity assertion in the conjecture (which is still open) is exactly 
an Ehrhart positivity question. 


5.2. Other Questions 


Many questions related to Ehrhart positivity remain open. We include a few below. 


5.2.1 Modified Bruns Question 


As we have discussed in Sect. 4.3, the answer to Bruns’ question of whether all smooth 
polytopes are Ehrhart positive is negative, where counterexamples are constructed 
for each dimension d > 3. Furthermore, we verify, with the help of polymake [2], 
that all smooth reflexive polytopes of dimension up to 8 are Ehrhart positive and 
that there exists a non-Ehrhart-positive smooth reflexive polytopes of dimension 9. 
However, we did not investigate smooth reflexive polytopes of higher dimensions. 
Therefore, one can ask: 


Question 5.2.1. Does there exist a smooth reflexive polytope of dimension d with 
negative Ehrhart coefficients, for any d > 10? 


Bruns’ question can be rephrased using the language of fans: For any smooth 
projective fan %, is it true that any polytope with normal fan & is Ehrhart positive. 
Since the answer is false, a weaker version of this question can be asked: 


Question 5.2.2. Is it true that for any smooth projective fan &, there exists one 
integral polytope P with normal fan & that is Ehrhart positive? 


5.2.2 h*-Vector for 3-Dimensional Polytopes 


Here instead of studying Ehrhart positivity question for families of polytopes in 
which dimensions vary, we focus on polytopes with a fixed dimension and ask the 
following question: 


Question 5.2.3. For each d, how likely is a d-dimensional integral polytope Ehrhart 
positive? 


Since integral polytopes of dimension at most 2 are always Ehrhart positive, dimen- 
sion 3 is a natural starting point. 
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Fig. 7 The plot of (47, 45) of 3-dimensional polytopes with | or 2 interior lattice points 


We have mentioned in the introduction that various inequalities for h*-vectors 
have been found. So we may use Formula (1.1) which gives a connection between 
the h*-vector and Ehrhart coefficients together with known inequalities to study 
Question 5.2.3. Note that in dimension 3, only the linear Ehrhart coefficient could 
be negative. Applying (1.1), we obtain that P is Ehrhart positive (equivalently the 
linear Ehrhart coefficient of P is positive) if and only if the h*-vector (ho, hf, h>, h3) 
of P satisfies 

11h + 2h} — hy + 2h > 0. (5.1) 


In [6], Balletti and Kasprzyk give classifications for 3-dimensional polytopes with 1 
or 2 interior lattice points, using which they extract all possible h*-vectors. Assume 
the number of interior lattice points is fixed to be | or 2. Applying (1.2), we obtain 

6 = lL and h3 = 1 or 2. Hence, only hj and h change. Balletti and Kasprzyk then 
plot all occurring pairs of (h}, 43) in [6, Figure5]. The black part of Fig.7 is their 
figure, which we modify to include a red line representing the inequality (5.1), where 
points below the red line arise from polytopes with the Ehrhart positivity property. 
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Note that in each part of the figure, the big triangular area is bounded by three known 
inequalities for h*-vectors. It is clear from the figure that these inequalities are far 
from optimal. Comparing the red line with the plotted data, one sees that a very high 
percentage of data points correspond to Ehrhart-positive polytopes. However, if we 
only look at the triangular region (without the data points), then the area below the red 
line has a much lower percentage of the region. Therefore, improving the inequality 
bounds for h*-vectors will be helpful in understanding the Ehrhart positivity problem, 
in particular, in giving a more accurate answer to Question 5.2.3. 
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Sarah K. Mason 


Abstract This article serves as an introduction to several recent developments in 
the study of quasisymmetric functions. The focus of this survey is on connections 
between quasisymmetric functions and the combinatorial Hopf algebra of noncom- 
mutative symmetric functions, appearances of quasisymmetric functions within the 
theory of Macdonald polynomials, and analogues of symmetric functions. Topics 
include the significance of quasisymmetric functions in representation theory (such 
as representations of the 0-Hecke algebra), recently discovered bases (including ana- 
logues of well-studied symmetric function bases), and applications to open problems 
in symmetric function theory. 


1 Introduction 


Quasisymmetric functions first appeared in the work of Stanley [117] and were for- 
mally developed in Gessel’s seminal article on multipartite P-partitions [44]. Since 
their introduction, their prominence in the field of algebraic combinatorics has con- 
tinued to grow. The number of recent developments in the study of quasisymmetric 
functions is far greater than would be reasonable to contain in this brief article; 
because of this, we choose to focus on a selection of subtopics within the theory of 
quasisymmetric functions. This article is skewed toward bases for quasisymmetric 
functions which are closely connected to Macdonald polynomials and the combi- 
natorial Hopf algebra of noncommutative symmetric functions. A number of very 
interesting subtopics are therefore excluded from this article, including Stembridge’s 
subalgebra of peak quasisymmetric functions [122] and its associated structure [16, 
17, 19, 20, 34, 81], Ehrenborg’s flag quasisymmetric function of a partially ordered 
set [34], colored quasisymmetric functions [69, 70], and type B quasisymmetric 
functions [25, 68, 100, 101]. This article also does not have the scope to address 
connections to probability theory such as riffle shuffles [120], random walks on 
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quasisymmetric functions [64], or a number of other fascinating topics. Hopefully 
this article will inspire the reader to learn more about quasisymmetric functions and 
explore these topics in greater depth. 

This article begins with an overview of symmetric functions. There are a num- 
ber of excellent introductions to the subject including Fulton [39], Macdonald [87], 
Sagan [108], and Stanley [119]. The remainder of Sect. 1 deals with the genesis of 
quasisymmetric functions and several important bases. Section 2 discusses the signif- 
icance of quasisymmetric functions in algebra and representation theory, while Sect. 3 
explores connections to Macdonald polynomials. A number of recently introduced 
bases for quasisymmetric functions are described in Sect. 4. Section 5 is devoted to 
interactions with symmetric functions. 


1.1 Basic Definitions and Background on Symmetric 


Functions 
Recall that a permutation of the set [n]: = {1,2,...,}1is a bijection from the set 
{1,2,...,} to itself. The group of all permutations of an n-element set is denoted 


G6,. Let 7 = 1 72---+ 7, € G, denote a permutation written in one-line notation. If 
7; > 741, theni isadescent of x. Ifa; > 7; and 1 <i < j <n, then the pair (i, j) 
is an inversion of 7. The sign of a permutation 7 (denoted (—1)”) is the number of 
inversions of 7. 

Let C[x), x2, ...,%X,] be the polynomial ring over the complex numbers C on 
a finite set of variables {x), x2,...,xX,}. A permutation 7 € G, acts naturally on 
F(X, X2,---5Xn) € Chy, %2,...,X,] by 


Tf (x1, X2, sen) = S Xm Xi5 ee 


Definition 1.1. The ring of symmetric functions in n variables (often denoted by A, 
or Sym,,) is the subring of C[x,, x2, ..., X,] consisting of all polynomials invariant 
under the above action for all permutations in Gy. 


This notion can be further extended to the ring Sym of symmetric functions in 
infinitely many variables. A symmetric function f € Sym is a formal power series 
Ff € C[[X]] (with infinitely many variables X = {x,, x2,...})suchthat f(x), xo, ...) 
= f (Xx,,Xx,,---) for every permutation of the positive integers. 

A partition X = (Ay, A2,.--, Ae) of a positive integer n is a weakly decreasing 
sequence of positive integers which sum to n. The elements \; of the sequence are 
called the parts and the number of parts is called the length of the partition (denoted 
£(A)). We write \ F n (or |A| = n) to denote “A is a partition of n.” 

Each partition A = (A), A2,..., Ae) of n can be visualized as a Ferrers diagram, 
which consists of n squares (typically called cells) arranged into left-justified rows 
so that the ith row from the bottom contains 4; cells. (Note that we are using French 
notation so that we think of the cells as indexed by their position in the coordinate 
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Fig. 1S is the Ferrers 


diagram for (5, 4, 2) and P S= PS ait 
is a filling of (5, 4, 2) 3 15/5] 2 
4}1)2)7/9 
Fig. 2 T is the composition 
diagram for (4, 2, 3) and F i eae 
is a filling of (4, 2, 3) 215 
715/5]3 


plane. This means the cell (i, j) is the cell in the ith column from the left and the 
jth row from the bottom. A Ferrers diagram in English notation places the rows so 
that the ith row from the top contains 4; cells, aligning with matrix indexing.) An 
assignment of positive integer entries to each of the cells in the Ferrers diagram of 
shape A is called a filling (see Fig. 1). 

A composition a = (Q1,Q2,...,Q¢) Of a positive integer n is a sequence of 
positive integers which sum to n. (It is sometimes necessary to allow 0 to appear 
as a part; a weak composition of a positive integer n is a sequence of nonnegative 
integers which sum ton.) The elements a; of the sequence are called the parts and the 
number of parts is called the /Jength of the composition. Write a — n to denote “a is 
a composition of n.” The reverse, a*, of acomposition a is obtained by reversing the 
order of the entries of a so that the last entry of a is the first entry of a*, the second to 
last entry of a is the second entry of a*, and so on. For example, if a = (4, 1, 3, 3, 2), 
then a* = (2, 3, 3, 1, 4). 

Each composition @ = (@, Q2,..., ag) of n can be visualized as a composition 
diagram, which consists of n cells arranged into left-justified rows so that the ith row 
from the bottom contains a; cells, again using French notation. (At times we will 
shift to English notation, but the reader may assume we are using French notation 
unless specified otherwise.) An assignment of positive integer entries to each of the 
cells in the composition diagram of shape a is called a filling. See Fig. 2 for an 
example of a composition diagram and a filling. 

The refinement order is a useful partial ordering on compositions. We say a < (3 
in the refinement ordering if 3 can be obtained from a by summing adjacent parts of 
a. For example, (2, 4, 1, 1,3) ~ (2,5, 4) and (2, 4, 1, 1, 3) ~ (6, 5) but (2,5, 4) & 
(6, 5) and (6, 5) & (2, 5, 4). 

Let x* be the monomial Ces .- oa For example, if A = (6, 4, 3, 3, 1), then 
x* = x?x3x3x}x5. One way to construct a symmetric function is to symmetrize such 
a monomial. The monomial symmetric function indexed by 4 is 


Miedo Xe 
m)(X) = ) Xj Xin °° Xj, 


where the sum is over all distinct monomials with exponents ;, A2,..., Ae. 

The monomial symmetric functions {m) | 4 - n} forma basis for the vector space 
Sym" of degree n symmetric functions. There are many elegant and useful bases for 
symmetric functions including three multiplicative bases obtained by describing the 


242 S. K. Mason 


basis element f,, and then setting fy = fy, fy, --- f\,. For example, the elementary 
symmetric functions {e, | A n} are defined by setting e, = mc), the complete 
homogeneous symmetric functions {hy | \ + n} are defined by setting h, = > Mm), 


AFn 
and the power sum symmetric functions {p, | \  n} are obtained by setting p, = 


Mn). Notice that the monomial symmetric functions are not multiplicative; that is, 
m)y is not necessarily equal to m),m), ---M),. 

Define a scalar product (a bilinear form ( f, g) with values in Q, sometimes referred 
to as an inner product) on Sym by requiring that 


(hy, My) = O\us 


where 6 is the Kronecker delta. This means the complete homogeneous and monomial 
symmetric functions are dual to each other under this scalar product. The power 
sums are orthogonal under this scalar product. This means (p, Pj.) = 6.Z,, where 
Zy = 1™ (m4!)2"" (m2)! - + -k’"* (my!) with m; equal to the number of times the value 
i appears in \. For example, 


Z(4,4,4,2,1,1) = 172!2'1!3°0!433! = 1536. 
(Note that u counts the number of permutations of cycle type  [14, 108].) 

Letw: Sym — Sym be the involution on symmetric functions defined by w(e,) = 
h,. (Note that this implies w(e,) = hy forall partitions A.) Thenw(p,) = (—1)""!p 


1.2) Schur Functions 


The Schur function basis is one of the most important bases for symmetric functions 
due to its deep connections to representation theory and geometry as well as its 
combinatorial properties. Schur functions are orthonormal under the scalar product 
described above and can be defined in a number of different ways. We begin with a 
combinatorial description, for which we will need several definitions. 

A filling of a partition diagram 2 in such a way that the row entries are weakly 
increasing from left to right and the column entries are strictly increasing from bottom 
to top is called a semi-standard Young tableau of shape . (see Fig. 3). The content 
of such a filling is the composition a = (a1, Q2,..., a), Where a; is the number of 
times the entry i apReer in the filling. The weight of a semi-standard Young tableau 
T is the monomial x’ = x}''x;? --- x;'*. The set of all semi-standard Young tableaux 
of shape A is denoted SSYT(A). A semi-standard Young tableau of shape in which 
each entry from 1 to n (where n = |A|) appears exactly once is called a standard 
Young tableau, and the set of all standard Young tableaux of shape is denoted 
SYT()). 
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3 2 3 2 3 3 3 
1} 1 1} 1 1/2 1/2 1/3} {1/38} |2]2] |2}3 


Fig. 3 Set of all semi-standard Young tableaux of shape (2, 1) whose entries are in the set {1, 2, 3} 


Definition 1.2. The Schur function s) (x1, x2, ..., X,) is the generating function for 
the weights of all semi-standard Young tableaux of shape with entries in the set 
{1,2,...,n}, that is 

S\(X1, X25 ..-,Xn) = >, ae 


T €SSYT(A) 


Here the sum is over all semi-standard Young tableaux whose entries are in the set 
[n]. Extend this definition to infinitely many variables by allowing entries from the 
set of all positive integers. 


Figure 3 shows that 
_ 2 2 2 2 2 2 
S21 (x1, X2, X3) = XP X2 + XP HZ + XpXZ A 2XpX2xZ + XY XZ + XZXZ + 2X3. 


Notice that sq) = man) = en, Since sqm) is constructed by filling a vertical col- 
umn with positive integers so that no entries repeat, and s, = h, since s, is con- 
structed by filling a horizontal row with weakly increasing positive integers. The 
Schur functions enjoy a large number of beautiful properties, including the celebrated 
Littlewood-Richardson formula for the coefficients appearing in the product of two 
Schur functions (which can also be computed algorithmically using the Remmel— 
Whitney rule [106]) and the Robinson—Schensted—Knuth Algorithm [75, 110] which 
maps bijectively between matrices with finite nonnegative integer support and pairs 
(P, Q) of semi-standard Young tableaux of the same shape. 

The Schur functions were classically described as quotients involving the Van- 
dermonde determinant and can be defined in a number of other ways. One method 
of construction that can readily be generalized to other settings is through Bernstein 
creation operators. 


Theorem 1.3. ([{125]) Define an operator By: Sym" —> Sym'*" by 


Bm = DO(-1)' Anse? 


i=0 


where f+: Sym — Sym is an operator defined by (f g,h) = (g, fth) forallg,h € 
Sym. Then for all tuples a € Z", 


Sq = Bo, Ba, -- - Ba,, (1). 
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Note that this method for constructing Schur functions is more general than the 
combinatorial method described above, because Bernstein creation operators define 
Schur functions indexed by tuples of nonnegative integers (weak compositions) rather 
than just partitions. 

Schur functions appear in many areas of mathematics beyond combinatorics. 
They correspond to characters of irreducible representations of the general linear 
group. Their multiplicative structure describes the cohomology of the Grassmannian 
of subspaces of a vector space. See the comprehensive texts by Fulton [39] and 
Sagan [108] for more details about Schur functions and their roles in combinatorics, 
representation theory, and geometry. 


1.3— Quasisymmetric Functions 


The ring Sym of symmetric functions is contained inside a larger ring of quasisym- 
metric functions, denoted by QSym, which can be thought of as all bounded degree 


formal power series f on an infinite alphabet x, x2, ... such that the coefficient of 
Oy 0 Ok: * bi a1 ..a2 Ok s 

i air aL f is equal to coefficient of Xip%_ 7%; in f for any sequence 

of positive integers 1 < jy < jy <--- < jy and any composition (a1, Q2,..., Ax). 


It is often convenient to restrict to n variables so that f € QSym,, if and only if the 
coefficient of xj''x5” ---x;" in f is equal to the coefficient of x7" xi? ---x¢* in f for 
any sequence of positive integers | < j, < jg <--- < jg <n and any composition 
(Q, (0 eee eae Qi). 

For example, the polynomial 


FS (1, X2, %3) = i. + axe + boxe 


is in QSym,, as is 
35 35 3.5 5,3 53 5.3 
G(X1, X2,.X3) = XP XQ A AXPXRZ A HQHZ A APH AXP HZ A XGXG, 


but 
5.3 


h(x1, X2, x3) = aie + xpxe + x5X3 
is not quasisymmetric in three variables since xexe does not appear, and neither 
do Aede and Aras The quasisymmetric functions in 1 variables are precisely the 
functions invariant under a quasisymmetrizing action of the symmetric group G,, 
introduced by Hivert [65, 66]. 

The origins of quasisymmetric functions first appeared in Stanley’s work on P- 
partitions [117]. Gessel introduced the ring of quasisymmetric functions through his 
generating functions for Stanley’s P-partitions [44]. (See Gessel [45] for a historical 
survey of P-partitions.) To be precise, let [m] be the set {1, 2,..., m} and let X be 
an infinite totally ordered set such as the positive integers. A partially ordered set 
(or poset), (P, <p), is a set of elements P and a partial ordering <p satisfying: 
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e reflexivity (Vx € P,x <p x), 
e antisymmetry (if x <p y and y <p x, then x = y), and 
e transitivity Gfx <p y and y <p z, then x <p 2). 


Write x <p yifx <p yandx # y. A P-partition of a poset P with elements [m] 
and partial order <p is a function f: [m]— X such that: 


(1) i <p j implies f(i) is less than or equal to f(j), and 
(1) i <p j andi > j (under the usual ordering on integers) implies f (7) is strictly 
less than f(j). 


Each permutation 7 corresponds to a totally ordered poset P, where 7) <,; 72 <x 
+++ <q Tm. It is these permutation posets that are used to construct Gessel’s funda- 
mental quasisymmetric functions F.,. To do this, give each P,-partition f a weight 
xf =[]x f(i) and sum the weights over all P,-partitions. 

For example, let 7 = 312 (written in one-line notation). Condition (1) implies 
that f(33) < fd) < f(2). Condition (2) implies f(3) < f(1). Therefore, f(3) < 
fC) < f(2), and the following table depicts the P,-partitions involving the subset 
{1, 2, 3} of X. 


fB)FD]FQ]) x7 
1] 2] 2 | xpx3 


1 | 2 | 3 |x,x2x3 


1} 3 4) 3 x1x5 


21343 xox5 


Therefore, the fundamental quasisymmetric function corresponding to 
am = (3, 1, 2) and restricted to three variables is 


2 2 2 
XX +X XIQH3 + XXZ + XQXZ- 


Note that this function depends only on the descent set of 7 and not on the underlying 
permutation; therefore, the indexing set for these generating functions is the set of 
all subsets of the set [7 — 1] together with the number m to indicate the degrees of 
the monomials. We use an ordered pair consisting of a capital letter together with m 
when indexing by sets. In this paper, we use Greek letters to denote compositions, 
but in other places, such as [43], capital letters are used. 


Definition 1.4. ({44]) Let L be a subset of [m — 1]. Then 


=— 5 XXX 
; ‘m 
Fram 1, %in i 


i) SinS--Sim 
ij<ijyi if jel 


246 S. K. Mason 


Every subset L = {Lj, L2,..., Lx} of the set [m — 1] corresponds to a composi- 
tion G(L) = (Ly, L2 — Li,--+ ,m — Lx), and therefore, the fundamental quasisym- 
metric functions are often indexed by compositions rather than sets. The fundamental 
quasisymmetric functions are homogeneous of degree m; the value m is apparent 
when the index is a composition @ since |a| = m. When the index is given by a 
subset, this value m must be specified. For example, if m = 4 then 


Fy,3},4(%15 X2, X3, X4) = ) Xj, Xin Xin Xiy 


11 Siz <i3 <i4 


2 2 2 2 
= Xj X23 + XP XQN4 + XPAZX4 + X1XIAZX4 + AXZAZX4. 
whereas if m = 5 then 


F(2,3},5(41, X2, X3, X4) = y Xi, Xin XizXigXis 
1] Siz <i3 <i4<is 


2 2 2 2 2 2 2 2 2 2 
= XY XIXZ + XY XINZX4 + AP AWNA + APA3Xg +X AANZAZ A XZ AZXG. 


The monomial quasisymmetric function, M,, in infinitely many variables 
X 1, X2,...} and indexed by the composition a = (a1, Q2,..., Ax), 1S obtained b 
y Pp y 
quasisymmetrizing the monomial x® = x/"'x3? ---+x;*. That is, 


_ Oy 1a OK 
Ma(x1,%2,...) = ) Xj Xj, Xj, 


1 <ig<++ <i 


This definition can be restricted to finitely many variables by requiring that i, <n. 
Note that m, = )°._, Ma, where @ is the partition obtained by rearranging the parts 
of a into weakly decreasing order. Every fundamental quasisymmetric function can 
be written as a positive sum of monomial quasisymmetric functions as follows: 


Fy = ° Mg. 


Bxa 


The Schur functions decompose into a positive sum of fundamental quasisymmet- 
ric functions; to describe this decomposition, we need one additional definition. Each 
standard Young tableau T has an associated descent set D(T) given by i € D(T) if 
and only if i + 1 appears in a higher row of T than i. Then 


SQ= = Fpcr),|\\- (1.1) 


TESYT(A) 


For example, if 4 = (3,2), then the standard Young tableaux of shape (3, 2) 
are shown in Fig. 4, with respective descent sets {3}, {2, 4}, {2}, {1,4}, and {1, 3}. 
Therefore, 
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4|5 3 3,4 2|5 2|4 
1/2/3 1|2|4 1/2 


or 


ol 
H 
w 
a 
bm 
w 
Or 


Fig. 4 Five standard Young tableau of shape (3, 2) 


83,2 = Fay5 + Fy.ays + Fay,5 + Friays + Fq3},5 
= P32. + Fog) + Fo3 + Fiat + Fi. 


Valuable information can be gained about symmetric functions by examining their 
expansion into quasisymmetric functions, especially into the fundamental quasisym- 
metric functions. For example, a symmetric function is said to be Schur-positive if 
it can be written as a positive sum of Schur functions. Schur positivity is important 
because of its deep connection to representations of the symmetric group. Assaf’s 
recently developed paradigm called dual equivalence [7] provides machinery to 
prove that a function is Schur-positive based on its expansion into the fundamentals 
and their connection to objects called dual equivalence graphs. The Eulerian qua- 
sisymmetric functions [113] are defined as sums of the fundamental quasisymmetric 
functions indexed by certain permutation statistics. Eulerian quasisymmetric func- 
tions are in fact always symmetric. Their generating functions are deeply connected 
to Euler’s exponential generating functions for the Eulerian polynomials. Eulerian 
quasisymmetric functions can also be used to refine a number of classical results on 
permutation statistics. We will not be able to address these topics in this brief survey 
article but encourage the interested reader to see [7, 113] for details. 


2 Algebra and Representation Theory 


Even before they were formally defined, quasisymmetric functions appeared natu- 
rally in algebraic settings. The Leibniz—Hopf algebra is the free associative algebra 
over the integers which in fact is isomorphic to the algebra of noncommutative sym- 
metric functions, which we shall define in Sect. 2.1. In 1972, Ditters claimed [30, 
Proposition 2.2] that the Leibniz—Hopf algebra is a free commutative algebra over 
the integers. This statement was later referred to as the Ditters Conjecture due to 
an error in the original proof and was then proved by Hazewinkel [60, 62] using 
combinatorial techniques and later by Baker and Richter [10] using methods from 
algebraic topology. Malvenuto and Reutenauer [90, Corollary 2.2] prove that QSym 
is a free module over Sym. In fact, Garsia and Wallach [41] further show that the 
quotient QSym,, over Sym, has dimension n!. Aval and Bergeron [9] prove that the 
quotient of Z[x), x2, ..., X,] modulo quasisymmetric functions in n variables with 
no constant term has Hilbert series )* C,t” where C,, is the nth Catalan number (see 
also [8] for the case with infinitely many variables). An explicit basis for the quotient 
space is given in [79]. 
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mMeid 


A®ABA A@A A@A 


idQm m R @ A + A @ R 


A@A 2 A GN a 


(a) Associativity property 


Fig. 5 Commutative diagrams for algebras 


C@C@C<~——_—_ C@C C@C 
A@id 
id@®e 
= = REC A C@R 
oe a 
1®@ 
Cec <—— C C 
(a) Coassociativity property (b) Counitary property 


Fig. 6 Commutative diagrams for coalgebras (Here 1 applied to an element c € C means c @ 1 
and 1@ applied to an element c € C means | ® c.) 


The ring of quasisymmetric functions (QSym) is an important example of a 
combinatorial Hopf algebra (discussed in Sect. 2.1). QSym is closely connected to 
Solomon’s descent algebra (described in Sect. 2.2) and plays a role in representations 
of the 0-Hecke algebra (see Sect. 2.3). 


2.1 Combinatorial Hopf Algebras 


The following definitions, leading to the description of a combinatorial Hopf algebra, 
closely follow the expositions in [47, 85]. 

Let R be acommutative ring with an identity element. An associative algebra over 
R is an R-module A together with a product (or multiplication) m: A@ A> A 
andaunitu: R > A satisfying associativity (m(m(a, b), c) = m(a, m(b, c))) anda 
unitary property which implies that the unit map commutes with scalar multiplication. 
To be precise, m and u are R-linear maps such that if id is the identity map on A 
and s is scalar multiplication, then the diagrams in Fig. 5 commute. 

A coalgebra over R is an R-module C together with a coproduct (or comulti- 
plication) A: C + C @C and a counit (or augmentation) ¢: C > R satisfying a 
coassociativity property and a counitary property so that when the directions of the 
arrows in Fig. 5 are reversed, the resulting diagrams (shown in Fig. 6) commute. 
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An algebra morphism is a map f: A— A’ from an R-algebra (A, m, u) to 
another R-algebra (A’, m’, u’) such that 


fom=m' o(f @ f)and fou=w'. 


A bialgebra is an algebra (8, m, u) and coalgebra (6, A, €) such that A and « are 
algebra homomorphisms. A bialgebra 6 with coproduct A is said to be graded if it 
decomposes into submodules Bo, 5,, ... such that 


(1) B= @pz0By, 
(2) BB; Cc Bi+;, and 
(3) A(Bn) Cc Bi+ janBi ® B;. 


Definition 2.1. A bialgebra (H, m, u, A, €) is a Hopf algebra if there exists a linear 
map S: H — H (called the antipode) such that 


mo(S@id)oAN=uoe=mo(id@S)od. 


A Hopf algebra 7 is said to be connected if Hy = R. When the ground ring R is in 
fact a field K , a character (sometimes called a multiplicative linear functional) of the 
Hopf algebra 7 is an algebra homomorphism from 7 to the field K. A combinatorial 
Hopf algebra is a graded connected Hopf algebra equipped with a character. 

Gessel [44] describes an internal (or inner) coproduct which takes QSym” (qua- 
sisymmetric functions of degree n) to QSym” @ QSym”. This internal coproduct 
corresponds to the internal coproduct on symmetric functions, taking p, to Py ® Pn. 
Malvenuto and Reutenauer [90] introduce an outer coproduct on QSym defined on 
the monomial quasisymmetric functions by 


k 
i=0 
For example, 
A(M312) = 1 @ M312 + M3 ® M2 + M3; @ M2 + M312 @ 1. 


Restricting this coproduct to symmetric functions takes p, to pp, ® 1+ 1® pp, and 
therefore, this coproduct is different from Gessel’s internal coproduct. Malvenuto 
and Reutenauer [90] and Ehrenborg [34] independently discovered the antipode map 
on QSym (with respect to the outer coproduct), proving that QSym is a Hopf algebra. 
See [61] for a thorough introduction to the Hopf algebra structure of quasisymmetric 
functions. 

If V is an R-module, let V* := Hom(V, R) be its dual R-module. Under certain 
finiteness conditions (which are true for the situations explored in this article), the 
duals of Hopf algebras are themselves Hopf algebras. In the language of Hopf alge- 
bras and their duality, the Ditters conjecture states that the Leibniz—Hopf algebra is 
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dual (as a Hopf algebra over the integers) to a free commutative algebra over the 
integers. 

The dual to QSym is the ring (or algebra) of noncommutative symmetric func- 
tions, denoted NSym. We take a moment to briefly describe some of the structure of 
NSym. For a thorough introduction to the topic through the lens of quasideterminants, 
see [43]; we typically follow their notation conventions. 

NSym can be thought of as a free associative algebra K (Aj, Ao, ...) generated 
by an infinite sequence of noncommuting indeterminates (A;),>1 over a fixed field 
K of characteristic 0. (We usually take K to be C, the complex numbers.) The 
noncommutative elementary symmetric functions are the indeterminates A,, and 


their generating function is 
A(t) = x t* Ax, 
k>0 


while the noncommutative complete homogeneous symmetric functions S; are defined 
by their generating function 


o(t) = ys ‘Sea iki. (2.1) 


k>0 


Note that this mirrors the relationship in Sym between the elementary and com- 
plete homogeneous symmetric functions, where if H(t) = 0,9 Mat” and E(t) = 
Y 20 ent”, then H(t) E(—t) = 1. Both of these basis analogues are multiplicative, 
meaning Sa = Sa,Sa,°+:Sa, and Ag = Aa, Aa, ++: Aa, for a = (1, Q2,... AK). 
The Ribbon Schur functions, which form a basis dual to Gessel’s fundamental basis 
for QSym, can be defined through quasideterminants. Two different candidates for the 
noncommutative analogue of the power sum symmetric functions will be described 
in Sect. 4.2. We use boldface letters for bases of NSym, lowercase letters for bases 
of Sym, and uppercase letters for bases of QSym. 

The forgetful map, frequently denoted by y: NSym — Sym, sends the basis ele- 
ment S,, to the complete homogeneous symmetric function hg, ha, «++ Na, Essen- 
tially, the forgetful map “forgets” that the functions don’t commute. This map can 
then be extended linearly to all of NSym and is in fact a surjection (but clearly not a 
bijection) onto Sym. 

Let C be a category of objects. An object T is a terminal object for the category C 
if for all objects X € C there exists a unique morphism X — T. Not every category 
has a terminal object, but if such a terminal object exists it is necessarily unique. 
Aguilar, Bergeron, and Sottile [1] introduce a canonical character Cg on QSym and 
describe what it does to the monomial and fundamental quasisymmetric functions. 
Equipped with this character, quasisymmetric functions are the terminal object in 
the category of combinatorial Hopf algebras. 


Theorem 2.2. ({1]) JfH is a combinatorial Hopf algebra with a character ¢, then 
there exists a unique homomorphism from (H, ¢) to (QSym, Gg) such that the homo- 
morphism on characters induced by the Hopf algebra homomorphism sends ¢ to Cg. 
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Theorem 2.2 helps to explain why quasisymmetric functions appear in so many 
different contexts throughout algebraic combinatorics. Examples for which the con- 
nection is well-understood include Rota’s Hopf algebra of isomorphism classes of 
finite graded posets [74] and the chromatic Hopf algebra of isomorphism classes of 
finite unoriented graphs [111]. Note that this mirrors the similar result stating that 
Sym is the terminal object in the category of cocommutative combinatorial Hopf 
algebras [1]. 


2.2 Solomon’s Descent Algebra 


Let ZG, be the group ring of permutations G,, over the integers and let Z6 = @n>0Z5,, 
be the direct sum of ZG, over all positive integers n. For each ao € G, let Des(c) be 
the descent set of o defined by Des(c) = {i | 1 <i <n—1,o0(i) > 0G + I}. To 


each subset L of {1, 2,..., — 1}, associate an element D;, of ZG, as follows: 
Dr = om CO. 
Des(o)=L 
The composition G(L) = (L, Lz — Li,...,n — Lx) (where k = |L|) is frequently 


used to index this descent basis. For example, let n = 4 and L = {2}. Then 
Dy = De.) = 1324 + 1423 + 2314 + 2413 + 3412. 


Let &,, be the linear span of the elements D; and endow © := @yn>0 Ln C ZS 
with a ring structure by setting om = O if 0 € G, anda € G,, such that m An. U 
is called Solomon’s descent algebra. Solomon [116] proves that & is a subalgebra 
of ZG. Gessel [44] shows that the algebra dual to the coalgebra QSym,, (endowed 
with Gessel’s internal coproduct) is isomorphic to Solomon’s descent algebra &,,. 

The set & (just as a set, not as the descent algebra) also admits a Hopf algebra 
structure. That is, Malvenuto and Reutenauer [89, 90] define a product and coproduct 
on ZG to prove that ZG is a Hopf algebra (called the Malvenuto—Reutenauer algebra) 
with & as a Hopf subalgebra. (This algebra is in fact isomorphic to the algebra FQSym 
of free quasisymmetric functions; see [32] for details.) Malvenuto and Reutenauer 
show (Theorem 3.3 in [90]) that & is dual to QSym, with the descent basis {D,} of X 
dual to the basis { F,,}. This means that the product on one of these bases determines 
the coproduct on the other, and vice versa. That is, if 


i 
FoFg = , Coahy 
yFlal+12l 


then comultiplication in & is defined by 


As(D,) = > Ch. gDa ® Dg. 
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This duality pairing also implies that the descent basis is isomorphic to the ribbon 
Schur basis for noncommutative symmetric functions since the ribbon Schur basis 
for NSym is dual to the fundamental basis for QSym. 


2.3 Representations of the 0-Hecke Algebra 


The representation theoretic significance of the fundamental quasisymmetric func- 
tions mirrors that of the Schur functions. We first describe the symmetric function 
case for ease of comparison. Recall that the symmetric group G, can be generated 
by adjacent transpositions s; = (j,i + 1) for 1 < i <n — 1 satisfying the following 
relations: 

s?=1lforl<i<n-1, 


Sj Sj4 18; = 87415; Sj 41 for 1 <i<n-—2, and 
sjS; = s;8; for |i — j| = 2. 


The Frobenius characteristic map is a map from characters of the symmetric 
group G,, to symmetric functions which are homogeneous of degree n. The Schur 
functions are the images of irreducible characters. 

The 0-Hecke algebra is a C-algebra generated by elements satisfying relations 
similar to the relations on the symmetric group generators described above. That is, 
H,,(0) is generated by elements 7;, 7>,..., T,—1 Satisfying: 


T? =T,; forl<i<n-1, 
T;Ti41T; = TiiT:Ti41 for 1 <i <n—2, and 
7;T; =T1;T, if |i — j| > 2. 


If o is a permutation in G, with reduced word o = 5;,58;,---8;,, then define T, € 

H,(0) by 

To = TiyTy,-*> Te: 
The 0-Hecke algebra H,, (0) is a specialization of the Hecke algebra H,,(q) at q = 0. 
(See Méliot [93] for further details on the Hecke algebra H,,(q) and its relationship 
to the 0-Hecke algebra H,,(0).) 

Norton [96] investigates the representation theory of H,,(0) and proves that there 
are 2"! distinct irreducible representations of H,(0), indexed by compositions of 
n. Let Go(H,,(0)) be the Grothendieck group of finitely generated H,,(0)-modules 
and G = ®n>0Go(Hn(0)) be the associated Grothendieck ring. (See Carter [23] for 
a thorough account of the representation theory of the 0-Hecke algebra and see 
Huang [71-73] for recent connections with flag varieties, the Stanley—Reisner ring, 


Recent Trends in Quasisymmetric Functions 253 


and tableaux.) Krob and Thibon [76] prove that G is isomorphic to the ring of qua- 
sisymmetric functions via a characteristic map F: G > QSym called the quasisym- 
metric characteristic. Let L, denote the irreducible representation of H,,(0) corre- 
sponding to a. Then F sends Ly, to the fundamental quasisymmetric function Fy. 


Theorem 2.3. ([31]) The map Ff, defined by F(Lq) = Fo, is a ring isomorphism 
between the Grothendieck group G of finite-dimensional representations of H,,(O) 
and the ring of quasisymmetric functions. 


The fundamental quasisymmetric functions therefore correspond to characters 
of irreducible representations of the 0-Hecke algebra. See Hivert [65, 66] to view 
this through the lens of divided difference operators. Similar representation theoretic 
interpretations can be ascribed to various other bases for quasisymmetric functions 
and will be discussed in the relevant sections. 


3 Macdonald Polynomials 


The Schur functions are uniquely determined by the following two requirements 
described on p. 305 of Macdonald [88]. 


(1) Let \ be a partition. Then 


S, =m) + ) Ky\ymp, 
[<r 


where ps < X if and only if wy + wo +--- + yj <A +A2+---A; for all j. 
(This partial ordering is called the dominance ordering.) Here the coefficients 
K),, are called the Kostka numbers, or Kostka coefficients. 

(2) (sy, Sy) = Oris 


Macdonald [86] generalized this construction to a two-parameter family of func- 
tions P, = P)(q, t) in Q(q, t) characterized by the following two requirements. 


(1) Let A be a partition. Then P, = m) + lower terms in dominance order. 
(2) (Py, Pudgt = Oif A A pw, where 


L(A) i qi 
(Prs Pudayt = OyyZr I] Tag 
i=l 


Notice that when g = ¢, the scalar product reduces to (p), Pu)g.g = O\uZ\ and 
so P\(q,q) = 5. Similarly, P\(qg,1) =m) and P\(1,t) =ey. (See p. 324 of 
Macdonald [88].) Therefore, the Macdonald polynomials are simultaneous gener- 
alizations of several different symmetric function bases. Macdonald polynomials 
also appear in connection with the Hilbert scheme of n points in the plane [59]. 
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There are several variations on the original definition of Macdonald polynomials, 
including the modified Macdonald polynomials, H ., obtained from P,, by certain sub- 
stitutions and motivated by their connection to the coefficients appearing in the Schur 
function expansion of Macdonald polynomials. Haglund conjectured and Haglund, 
Haiman, and Loehr proved [50, 51, 54, 55] a combinatorial formula for the Mac- 
donald polynomials H, using statistics on fillings of partition diagrams. To describe 
this formula, we introduce several pertinent definitions. 

Recall that a filling 7: ~ — Z* is a function from the cells of the diagram of 
a partition jz to the positive integers. The reading word of the filling is the word 
obtained by reading the entries of the filling from top to bottom, left to right. 

The major index and inversion statistic on permutations can be generalized to 
statistics on fillings of Ferrers diagrams. Let s be a cell in the partition diagram ju 
and let South(s) be the cell immediately below s in the same column as s. Define 


Des(a, 14) = {s € | o(s) > o(South(s))}. 


(No cell in the bottom row of jz: can be in Des(a, j).) Let leg(s) be the number of 
cells above s in the same column as s and let arm(s) be the number of cells to the 
right of s in the same row as s. Then 


maj(o,)= >> (leg(s) +1). 


seDes(a, /1) 


Let u, v, w be three cells in the diagram of j such that uv and v are in the same row 
of j4 with v strictly to the right of uv and w = South(u) as shown: 


ul. v 


Any collection of three cells arranged in this way is called a triple. Define an orien- 
tation on the cells in a triple of a filling o of ju by starting with the cell containing the 
smallest entry and moving in a circular motion from smallest to largest. (If two entries 
are equal, the one which appears first in the reading word is considered smaller.) If 
the resulting orientation is counterclockwise, the triple is called an inversion triple 
(see Fig. 7). Two cells u, v in the bottom row are also considered an inversion triple 
if v is strictly to the right of wu and a(u) > o(v). The total number of inversion triples 
in a filling o of a partition ps is denoted inv(a, j1). 

For example, the filling in Fig. 8 has descent set Des = {(1, 3), (2, 3), (2, 2)}, 
where cells are indexed by (column, row) to mimic the (x, y) Cartesian coordinates. 


Fig. 7 First two triples are ead Lae Las Sane 
inversion triples; the third 2 c ! u z es s 
and fourth are not 
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Fig. 8 A filling of the 518 

partition (4, 3, 2) with 

reading word 582613371 2/6/1 
3/3}; 7]1 


The major index for this filling is 1 + 1+ 2 = 4 and inv(o, w) = 3+ 2 =5, since 
there are three inversion triples in the bottom row and two additional inversion triples. 


Theorem 3.1. ((54, 55]) Let us be a partition of n. Then 


H,,(X; qs t) — > x glo) malo) 


o: p>Zt 


Theorem 3.1 provides a straightforward method for computing Macdonald poly- 
nomials. This formula could potentially be used to find a product rule for Macdonald 
polynomials utilizing tableau constructions, although Yip recently found an elegant 
combinatorial rule for multiplying Macdonald polynomials [124] using the alcove 
walk model introduced by Ram and Yip [104]. 


3.1 Quasisymmetric Decomposition of Macdonald 
Polynomials 


Macdonald polynomials can also be described as sums of fundamental quasisym- 
metric functions with coefficients in g and f. 


Theorem 3.2. ((52, 54]) Let ys be a partition of n. Then 


H, (Xx; q; t= > ght pain) Fpes(g-')s 
BeGn 


where each permutation (3 in the sum corresponds to the standard filling of {4 with 
reading word 3 and Des(3~') is the usual descent set of the permutation 3~| obtained 
by taking the inverse of (3. 


For example, if 4 = (2, 1), the following Table demonstrates that 
Ay (Xi q.)=Fy+@toFi+@+0Fot+ qth. 


This expansion of the Macdonald polynomials into fundamental quasisymmetric 
functions paves the way for new approaches to long-standing open questions. For 
example, Macdonald [86] conjectured that the coefficients in the expansion of A, 
into Schur functions are polynomials in g and t with nonnegative integer coefficients. 
Haiman [59] proved this by showing that H, 18 the bigraded Frobenius character of 
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Permutation 3 123 132 213 231 312 
(Reading word of filling) 


Filling of ju 


a doubly graded G,,-module, but this approach did not provide an explicit combina- 
torial formula for the coefficients. Assaf’s dual equivalence [6, 7] provides another 
potential approach to Schur positivity which makes use of the decomposition of a 
symmetric function into fundamental quasisymmetric functions. 

The Hall—Littlewood polynomials are a one-parameter specialization of Macdon- 
ald polynomials introduced by Littlewood as a symmetric function realization of the 
Hall algebra [82]. Several different candidates for quasisymmetric Hall—Littlewood 
polynomials have recently been proposed. See Hivert [66] for an analogue in NSym 
and its QSym companion, and see Novelli, Thibon, and Williams [97] for a dif- 
ferent noncommutative analogue. Connections between these two approaches are 
studied in Novelli, Tevlin, and Thibon [98]. See also Haglund, Luoto, Mason, and 
van Willigenburg [57] for another quasisymmetric analogue. 


3.2 Quasisymmetric Schur Functions 


Haglund’s formula (Theorem 3.1) to generate the Macdonald polynomials using 
statistics on fillings of partition diagrams is generalized in [56] to fillings of weak 
composition diagrams in order to generate the nonsymmetric Macdonald poly- 
nomials introduced and initially developed by Cherednik [24], Macdonald [87], 
Opdam [99], and Sahi [109]. When these polynomials are specialized tog = t = ~, 
the resulting polynomials, called Demazure atoms due to their connections to 
Demazure characters, form a basis for all polynomials. The Demazure atoms decom- 
pose the Schur functions in a natural way, and their generating diagrams satisfy 
a Robinson—Schensted—Knuth-style algorithm [91]. Type A key polynomials [78, 
105] are positive sums of Demazure atoms [92]. Summing the Demazure atoms over 
all weak compositions which collapse to a fixed composition when their zeros are 
removed produces a new collection of quasisymmetric functions, called the gua- 
sisymmetric Schur functions, which we now formally define using fillings of com- 
position diagrams [57]. 

Let a be a composition of n. If T is a filling of the composition diagram a 
(written in English notation) satisfying the following properties, then T is called a 
semi-standard reverse composition tableau, abbreviated SSRCT. 


(1) The entries in each row weakly decrease when read from left to right. 
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(2) The entries in the leftmost column strictly increase when read from top to bottom. 

(3) (Triple Rule) Ifk > j and T(,k) > T@ +1, j) (for cells @, k) and @ + 1, /)), 
then (i+ 1,k)isacellinaand T+ 1,k) > TG +1, /). (Here, if there is no 
cell at coordinate (i, j), set T(i, 7) = 0.) 


The set of all semi-standard reverse composition tableaux of shape a is denoted 
SSRCT(q). The weight of a semi-standard reverse composition tableau T, denoted 
XT, is the product over all i of oo, where #(i) is the number of times i appears in 
T. 


Definition 3.3. The quasisymmetric Schur function S, is defined by 


S(O=. Sx, 


TeSSRCT(a) 


Quasisymmetric Schur functions form a basis for QSym and are closely related 
to Schur functions. In fact, the quasisymmetric Schur functions, when summed over 
all rearrangements of a given partition, produce the Schur function indexed by this 


partition [57]. That is, 
Sd = Ss Sa, 
a=) 


where @ is the partition obtained by arranging the parts of a into weakly decreasing 
order. 

For example, the four semi-standard reverse composition tableaux of shape (2, 1) 
are 


T/t) (1 ]a) [2)9) onal [2 
3 3 3 


producing the quasisymmetric Schur function 
2 2 2 
S21 (X1, X2,.X3) = XP XQ + XP XZ + Xp X2HZ + XZX3. 


The four semi-standard reverse composition tableaux of shape (1, 2) are 


1 1 i)... andl 2 
2/2] [3[2] [3/3 33 


producing the quasisymmetric Schur function 
= 2 2 2 
S12(X1, X2,X3) = XyXZ HF Xp XIKZ + XYXz + x0NG. 
Together these sum to 821 (%1, X2, x3); that is, 


821 = So, + Sy. 
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The quasisymmetric Schur functions expand positively in the fundamental basis 
for QSym. The set of all standard reverse composition tableaux of shape a, abbre- 
viated SRCT(q), is the subset of SSRCT(q) consisting of the semi-standard reverse 
composition tableaux in which each of the positive integers in the set {1, 2,..., |a|} 
appears exactly once. Each standard reverse composition tableau T has a descent set 
Des(T) defined by 


Des(T) = {i |i + 1 appears weakly right of i} C [n — 1]. 


Theorem 3.4. ((57]) The quasisymmetric Schur function S,, decomposes into the 
fundamental basis for quasisymmetric functions as follows: 


So = > Fpes(T),|a| + 


TeSRCT(a) 


For example, the three standard reverse composition tableaux of shape (2, 1, 3) 


are 
ait). j2ia. wa (2a 
4 4 3 
6 [5/2 6/5 [3 6[ 5/4 


The descent sets are, respectively, {1, 3, 4}, {2,4}, and {2, 3}. This implies that 


S213 = Fri 3.4.6 + Fo.4y.6 + F2,3),6- 
Tewari and van Willigenburg [123] introduce a collection of operators {7; a 
on standard reverse composition tableaux (which satisfy the same relations as the 
generators 0 ae described in Sect. 2.3) to produce an H,,(0)-action on standard 
reverse composition tableaux of size n. 

In particular, for T € SRCT(qa) for some compositiona F nand1 <i<n-—1, 
entries i andi + | are said to be attacking if they are in the same column of T or 
they are in adjacent columns of T with i + 1 appearing to the right of 7 in a strictly 
lower row. The operators 7; for 1 < i <n — 1 are defined as follows, where s;(T) 
interchanges the positions of entries 7 andi + 1. 


T if i ¢ Des(T), 
wi(T) = +0 if i € Des(T), i andi + 1 are attacking, and 
s(T) ifi € Des(T), i andi + 1 are nonattacking. 


Extend these operators to all of G, by setting 7, = 7;,7;,---7;, when g = 
Sj, Si, +++ Sj, 18 any reduced word for a. Define a partial order <, on SRCT(a) by 
setting 7; <, T> if and only if 7,(7\) = T> for some permutation o € G,,. Extend 
X, toa total order <!, arbitrarily and let V7, be the C-linear span of all T; ¢ SRCT(a) 
such that 7; >!, Tj. 
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Theorem 3.5. ([123]) Jf7, € SRCT(q) is the minimal element under the total order 
x! then Vz, := § is an H,(0)-module whose quasisymmetric characteristic is the 


quasisymmetric Schur function Sq. 


A simple composition is a composition a = (a, Q2,..., a) such that if a; > 
a; =>2and1 <i <j < £, then there exists an integer k satisfying i < k < j such 
thata, =a; — 1. 

Tewari and van Willigenburg [123] prove that S is an indecomposable H,,(0)- 
module if and only if a is simple. These results lead to the introduction of a new 
basis for quasisymmetric functions called the canonical quasisymmetric functions 
{C,}q and a branching rule for the S which is analogous to the classical branching 
rule for Schur functions [108]. 

The product of a quasisymmetric Schur function and a Schur function expands into 
the quasisymmetric Schur function basis through a rule which refines the Littlewood— 
Richardson Rule [58] but a formula for the coefficients appearing in the product of 
arbitrary quasisymmetric Schur functions is unknown. See [85] for a thorough intro- 
duction to quasisymmetric Schur functions and their closely related counterpart, the 
Young quasisymmetric Schur functions. The Young quasisymmetric Schur functions 
are obtained from the quasisymmetric Schur functions by a simple reversal of the 
indexing composition and the variables, but at times the Young quasisymmetric Schur 
functions are easier to work with due to their compatibility with semi-standard Young 
tableaux (rather than reverse semi-standard Young tableaux). 


4 Quasisymmetric Analogues of Symmetric Function Bases 


Quasisymmetric functions play a major role in answering important questions about 
symmetric functions. Analogues in QSym of classical bases for symmetric functions 
aid in this pursuit by providing a dictionary to translate between Sym and QSym. 
We have already discussed a quasisymmetric analogue of the monomial symmetric 
functions as well as two different quasisymmetric analogues of the Schur functions. 
We now introduce another natural quasisymmetric analogue of the Schur function 
basis as well as a quasisymmetric analogue of the power sum basis. 


4.1 Dual Immaculate Quasisymmetric Functions 


Berg, Bergeron, Saliola, Serrano, and Zabrocki [12] generalize Bernstein’s creation 
operator construction of the Schur functions to obtain a basis for NSym called the 
immaculate basis and denoted Z,,. The dual basis in QSym, called the dual immac- 
ulate quasisymmetric functions, can be generated by fillings of tableaux as follows. 

Let F: a > Z* be a filling of a composition diagram a with positive integers 
such that the sequence of entries in each row (read from left to right) is weakly 
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Fig. 9 Four immaculate 
tableaux of shape (2, 1, 3) ; 218 ae ; al cae 
and weight x1 x2x3x4%5 il3 13 ils ila 


increasing and the sequence of entries in the leftmost column (read from bottom to 
top) is strictly increasing. Then F is said to be an immaculate tableau of shape a. 
(Note that since we are using French notation our definition varies slightly from the 
definition in [12] but produces the same diagrams modulo a horizontal flip.) The 
weight of an immaculate tableau U, denoted x”, is the product over all i of a, 
where #(i) is the number of times 7 appears in U. 


Definition 4.1. ({12]) Let a be a composition. The dual immaculate quasisymmetric 
function Z* is given by 
n=. 
U 


where the sum is over all immaculate tableaux of shape a. 


For example, the coefficient of X1X2X3XX5 in Ty 13 is 4 since there are four 
immaculate tableaux of shape (2, 1, 3) and weight X1XIXZX{ Xs. (These immaculate 
tableaux are given in Fig. 9.) 

The following theorem provides a formula for the expansion of the Schur functions 
into the dual immaculate quasisymmetric functions. 


Theorem 4.2. ([{12]) Let X be a partition of length k. Then 


oT 


_ ov 
x= > (-) Ty, 41 Ot sda 2-02-01 Aoy ETE? 
o 


where (—1)° is the sign of o and the sum is over all permutations 0 € ©, such that 
Xo, ti- oj) > Oforalll <i <k. 


For example, 
__ T* ok 
$321 = Ly) — Ly). 


Note that the coefficients are not always nonnegative and further the compositions 
indexing the terms appearing in this expansion are not merely rearrangements of 
the partition \ as is the case in the quasisymmetric Schur expansion of the Schurs. 
However, the beauty of the connection to Schur functions is more readily apparent 
in the dual, since applying the forgetful map to an immaculate function produces the 
corresponding Schur function. That is, y(Za) = Sq. 

Grinberg recently proved Zabrocki’s conjecture that the dual immaculate qua- 
sisymmetric functions can also be constructed using a variation on Bernstein’s cre- 
ation operators [46]. The dual immaculate quasisymmetric functions expand into 
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positive sums of the monomial quasisymmetric functions, the fundamental qua- 
sisymmetric functions, and, recently shown in [3], the Young quasisymmetric Schur 
functions. The latter expansion is not at all obvious given the very different methods 
used to generate these two bases and therefore provides further justification that both 
of these families of functions are interesting and natural objects of study. 

Like quasisymmetric Schur functions, dual immaculate quasisymmetric functions 
correspond to characteristics of certain representations of the 0-Hecke algebra [13], 
but for the dual immaculate quasisymmetric functions these representations are inde- 
composable. In particular, let MM, be the vector space spanned by all words on the 
letters {1,2,..., €(@)} such that the letter j appears a; times. Define an action of 
the 0-Hecke algebra on words by 


tins w Wi = Wi+1 
Si(W) Wi < Wit, 


where 5;(w) = W1W2- ++ Wj-1Wj41W; Wi+2° ++ Wy. Note that this is isomorphic to the 
induced representation 


H, (0) 
Ind Ff, | (0)® Hay ()®-®Hay,,, (0) Lar ®@ Laz ® +++ ® Lay); 


where L; is the one-dimensional representation indexed by the composition (k). A 
word w in which the first instance of j appears before the first instance of j + 1 is 
called a Y-word. The 0-Hecke action defined above cannot move a j + | to the right 
of a j, so the subspace Ni, of M, spanned by all words which are not -words is a 
submodule of M,. 


Theorem 4.3. ({13]) The characteristic of Va := Mo/Nza is the dual immaculate 
quasisymmetric function indexed by a. In other words, F ((Va]) = T;. 


Bergeron, Sanchez-Ortega, and Zabrocki found a Pieri rule (first conjectured 
in [12] and proved in [18]) for the product of a fundamental quasisymmetric function 
and a dual immaculate quasisymmetric function, and much is known about the mul- 
tiplication of the immaculate basis. However, multiplication rules in full generality 
for the dual immaculate quasisymmetric functions are still largely unknown. 


4.2 Quasisymmetric Analogues of the Power Sum Basis 


The power sum symmetric functions (defined in Sect. 1.1) are eigenvectors for the 
omega involution w; that is, w(p,) = €,p,, where e, = (—1)"~) [119]. Power 
sum symmetric functions are also helpful in computing characters of the symmetric 
group via the Murnaghan—Nakayama rule [94, 95]. 

Malvenuto and Reutenauer [90], through the Hopf algebraic dual, NSym, of 
QSym, introduce a quasisymmetric analogue of the power sum symmetric functions, 
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also obtained independently by Derksen [29] using a similar process but with a com- 
putational error which leads to a different formula. To understand their construction, 
we recall several facts about generating functions for symmetric and noncommutative 
symmetric functions. The complete homogeneous symmetric functions, elementary 
symmetric functions, and power sum symmetric functions (in n variables) can be 
defined through their generating functions 


n 


n k 
HO=)> hat =|] — -E@=)> as Le + x;t), and P(t) = ae. 


d>0 i=l : k>0 k>1 


The relationship between these is given by Newton’s formula: 


d 
=< et) = P(t)E(t), 
t 


which is equivalent to 


d 
a EO) = BOP O. 


In their seminal work on noncommutative symmetric functions, Gelfand, Krob, 
Lascoux, Leclerc, Retakh, and Thibon [43] define a noncommutative analogue of 
the complete homogeneous symmetric functions (denoted S;) by describing their 
generating function (see Sect. 2.1) and requiring they satisfy the multiplicative prop- 
erty. 

They then utilize this approach to construct two analogues of the power sums in 
NSym by requiring that the generating functions satisfy the appropriate analogues 
of Newton’s formula. Noncommutative power sum symmetric functions of the first 
kind, denoted W;, are defined by 


d 
k-1 
th= tw —o(t) =oa(t)wt d Wy = Va, Va ++: Va, 
w(t) 2 be Fa(t)=o(Q¥), an Was 


where o(f) is as defined in Eq. 2.1. Similarly, noncommutative power sum symmetric 
functions of the second kind, denoted ®,, are defined by 


® 
a(t) = exp (x “) and ®y = Py, Pq, --- Pa,- 
k>1 


Taking the Hopf algebraic duals of these noncommutative power sum bases pro- 
duces two different quasisymmetric analogues of power sums. We use W and ® as 
notation for these to emphasize their relationship with their noncommutative duals. 
The dual of the noncommutative power sum basis of the first kind is defined [11] by 
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where the ordering used is the refinement partial order (so that a > (@ if a is 
coarser than (3) and m(a, 2) is given by the following process. First define 7(a) = 


gees a a;. Then for a a refinement of (3, set 7(a, 3) = Abe ma), where 


i=l Hi 
a” consists of the parts of « that combine to 4}. 

For example, W312 = dd -2- 3)(=45 M312 + 5 Me + sta M33 + x45 Mo), 
which simplifies to 


1 2 1 
W35=M M. M Mg. 
312 312+ 42 + 3 33 + DD 6 


Similarly, a formula for quasisymmetric power sums of the second kind is also 
given in terms of the monomial quasisymmetric functions. 


_ Ms 
a=) Fa By’ 


a>p 


where the ordering used is again the refinement partial order, and the function 
f(a, 2) is given by the following process. Assume 3 = (3), 3o,..., Gx). Write 
@ as a concatenation aq ---a of compositions a where a” / ;. Then 
f(a, B) = €(a)!.-- £(a® JI. 
For example, ®3;2. = (1-2- 3)(q45 M12 + 37M + M33 + Mo), which 
simplifies to 
@312 = 6M312 + 3M4z + 3M33 + Mo. 


This formula differs from that of Malvenuto and Reutenauer [90] (who use the 
notation P,, instead of ®,,) only by a constant. This constant ensures that the ®, 
refine the symmetric power sums so that 


Py = se Pq, 
a= 


which is not true for the P,. 

The reader might wonder about the duals of the elementary and complete homo- 
geneous symmetric functions. In fact, the noncommutative complete homogeneous 
symmetric functions are dual to the monomial quasisymmetric functions, while the 
noncommutative elementary symmetric functions are dual to the “forgotten” basis 
for quasisymmetric functions, whose combinatorial structure is largely unknown. 

Recall that the fundamental quasisymmetric functions satisfy the following rela- 
tionship to monomial quasisymmetric functions: 


Fo = 97 Ms, 


Bxa 
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where 3 < a again means that ( is arefinement of a. Hoffman [67] studied a variation 
on the fundamental basis called the essential quasisymmetric functions E.,, obtained 
by reversing the inequality in the above equation so that 


Ea = >) Mp. 


Bra 


Summing over all coarsenings of a is a natural thing to do because of what the 
antipode map does to the monomial quasisymmetric functions: 


S(Mq) = (-1)E,. 


Multiplication in the essential basis follows the same rules (modulo a sign) as mul- 
tiplication in the monomial basis. 


4.3 The Shuffle Algebra 


The shuffle algebra is a Hopf algebra (whose multiplicative structure is given by 
an operation called a shuffle) which is in fact isomorphic as a graded Hopf algebra 
to QSym (over the rationals). More details on the shuffle algebra and the closely 
related concept of Lyndon words can be found in Reutenauer [107], Lothaire [84], 
or Grinberg-Reiner [47]. 

Let A be a totally ordered set, which we will call an alphabet. A word of length n 
is an ordered string w = w,w2--- w, of elements of A. Let A* be the set of all words 
on the alphabet A. For the purposes of this section, we will take the alphabet to be 
the positive integers, as is done in [60]. When A is taken to be the positive integers, 
the degree (|u|) of a word u in A* is the sum of its letters rather than the number 
of letters. The shuffle, u LL! v, of two words u = uju2--- ug and v = vj v2--- Ve in 
A* is the sum of all words in A* of length k + € formed from the letters of u and 
the letters of v such that for all 7, u; appears before u;,, and v; appears before v;+. 
Multiplicities will occur if a letter appears in both uw and v. If a letter appears more 
than once within one of the words u or v, simply consider each occurrence as a 
distinct letter by applying a different subscript to each appearance of a given letter. 
This product is associative and can therefore be extended to the shuffle product of a 
finite number of words. (It is called a shuffle because it resembles the interleaving 
method used to shuffle a deck of cards.) For example, 


23 LL 12 = 2312 + 2132 + 2123 4 1223 4+ 1223 + 1232. 


Shuffles in fact guide the multiplication of quasisymmetric power sums of both 
types. Let a; equal the number of parts of size j in a, b; equal the number of parts of 
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os 


size j in 3, and let a - @ denote their concatenation. Define C(a, 3) = [| j ( 
so that C(a, 3) = Za.8/(Zaz8)- 


Theorem 4.4. ([{11]) Let a and 2 be compositions. Then 


1 1 
A w d ,0,= ,,. 
>= Cla, B) a = °= Cla, B) 2% 


yea lu 2 yea LU B 


The shuffle algebra K (A) (where K is a commutative ring with unit) is the set of 
all linear combinations over K of words on an alphabet A, endowed with this shuffle 
product. There are a number of distinct proofs that this algebra is isomorphic to QSym, 
including those of Hazewinkel [60] and Hazewinkel-Gubareni-Kirichenko [63]. Note 
that Theorem 4.4 in fact implies that the shuffle algebra is isomorphic to QSym. To 
see this, distribute the C(a, (3) in the first equation in Theorem 4.4 so that: 


YoWs yx My 


Za %B Za-B 


yea lu B 


Then map from the shuffle algebra to QSym via the map a +> “=. Extend this map 
linearly to an isomorphism between the shuffle algebra and QSym. 

We now discuss the algebraic structure of QSym. First we shall see that QSym 
over the rationals is a polynomial algebra in the quasisymmetric power sums. Then 
we describe Hazewinkel’s polynomial generators for QSym over the integers. 

Let w € A* be a word on the alphabet A. Then a proper suffix of w is a word 
v € A* such that there exists a nonempty u € A* such that w = uv. A prefix of w 
is a word u € A* such that w = uv. Let <, be a total ordering on A* defined as 
follows. Let u = uju2--- ug and v = v1 v2--- Um. If u is a prefix of v then u <, v. 
Otherwise let j be the smallest positive integer such that u; A v;. If u; > v; then 
u >, v. Otherwise u <, v. 


Definition 4.5. A Lyndon word is a nonempty word w € A* such that every 
nonempty proper suffix v of w satisfies w <, v. Let £L denote the set of all Lyn- 
don words. 


For example, the words 1324, 1323, and 11213 are Lyndon words while the words 
4132, 3241, 2332, and 2233 are not. The shuffle algebra is freely generated over the 
rationals by the Lyndon words. 


Theorem 4.6. ([103]) Every element of K (A) can be uniquely expressed as a poly- 
nomial in the Lyndon words. In other words, the shuffle algebra K (A) is the polyno- 
mial algebra in the Lyndon words. 


One can think of Theorem 4.6 (commonly known as Radford’s Theorem) as the 
statement that for any vector space basis whose elements are indexed by words in 
A* and whose multiplication is given by shuffles, each basis element can be written 
as a polynomial in basis elements indexed by Lyndon words. 
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For example, w = 321 is not a Lyndon word, but 32; can be expressed as a 
polynomial in quasisymmetric power sums indexed by Lyndon words. That is, 


W301 = Wy) - Wo V3 — Wo3 - Wy — V3 - Wyn + Wj93. 


Radford’s theorem implies that quasisymmetric power sums indexed by Lyndon 
words form an algebraically independent generating set for QSym over the rationals 
(see [11, 47, 90, 107] for further details). 

We now describe Hazewinkel’s polynomial generators for QSym, which are 
indexed by a subset of Lyndon words. The ring Z[x), x2,...,] is endowed with 
a well-known A-ring structure via 


xj i=l 
Xi Xj) = d 
()) 0 i>l, 
for j = 1,2,...,. Define a total ordering on compositions called the well-ordering 


(weight, length, lexicographic) by: 


(1) If la] > |G, then a > yz B. 
(2) If |a| = |G| and (a) > £(3), then a > yz B. 
(3) If la] = |Gl, Ca) = £(8), and a >jex 3, then a Swi (. 


For example, 
523 > will 11213 > wil 323 > will 143. 


Hazewinkel proves [62] that applying .,, to the monomial quasisymmetric func- 
tion indexed by a Lyndon word a produces 


An(Ma) = Mar + (smaller), 
where a*” denotes concatenation of a with itself n times and (smaller) is a Z-linear 
combination of monomial quasisymmetric functions which are wll-smaller than a*”. 
For example, 


A2(M1,2)) = M1212 + some subset of the set VU W UY, 


where 
V = {all words with weight < 5}, 


W = {all words of weight 6 and length < 3}, 


and 
Y = {1122, 1113, 1131}. 
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Theorem 4.7. ((62]) Let eLYN be the set of all Lyndon words u = u,u2--- Um 
such that gcd{u,, U2, ..., Um} = 1. Then the set {A,(My) }ucetyn foralin € N freely 
generates the ring of quasisymmetric functions over the integers. 


Although the monomial quasisymmetric functions are not multiplicative, The- 
orem 4.7 provides a way to construct a multiplicative generating set. Therefore, 
QSym is a polynomial algebra over the integers in the set {A, (Ma) }acetyn- Note 
that Theorem 4.7 therefore implies the Ditters conjecture. 


5 Connections to Symmetric Functions and the Polynomial 
Ring 


This section discusses several recent developments connecting quasisymmetric func- 
tions to important open problems within symmetric functions and the polynomial 
ring. We focus our scope to three topics: chromatic quasisymmetric functions, transi- 
tions from QSym to Sym, and liftings of QSym bases to the polynomial ring. We will 
unfortunately not be able to address the Eulerian quasisymmetric functions [113], 
which are in fact symmetric despite their definition in terms of quasisymmetric func- 
tions. See [113] for a wonderful introduction to these fascinating objects of study, 
including the important definitions and theorems as well as the research avenues they 
introduce. We also regretfully omit the recently developed theory of dual equivalence; 
see [7] for information about this new paradigm and how to use it. 


5.1 Chromatic Quasisymmetric Functions 


Let G = (V, E) bea graph with vertices V and edges E and let S be a subset of the 
positive integers P. A proper S-coloring of G is a function «: V — S such that if 
two vertices i and j are adjacent (i.e., {i, j} € E), theni and j are assigned different 
colors (i.e., (i) 4 K(j)). The chromatic number x(G) is the minimum number of 
colors (size of S) necessary to construct a proper S-coloring of G. 

It is natural to ask how many proper {1, 2, ..., m}-colorings exist for a graph G; 
this number is denoted yg (m). Itis anonnegative integer when m is a positive integer, 
and it is a polynomial called the chromatic polynomial when m is an indeterminant. 
Stanley generalized this notion [118] to construct a symmetric function generated 
from the set C(G) of all proper P-colorings of G as follows: 


Xo) = Do xn 


KEC(G) 
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where x = (x1, X2,...) is a sequence of commuting variables and x,, = Trev Xv): 
Notice that plugging in x; = | for alli produces the chromatic polynomial X¥g(1”) = 
xG(m). For example, the path P; on three vertices has chromatic symmetric function 


2 2 a 2 2 2 
X p,(X1, X2, X3) = XP X2 + XPXZ + XGHXZ + HAZ + HXZ + XXX + OX1X23, 


and Xp, (1, 1, 1) = 12, the number of proper colorings of P3 with 3 colors. 
Recall that if a function has positive coefficients when expanded in a basis B, 
then it is said to be B-positive. For example, the elementary symmetric functions are 


Schur-positive since 
En= ) Ky Sy, 
be 


where Ky is the number of semi-standard Young tableaux of shape j’ and content 
A. One significant open question about chromatic symmetric functions relates to 
positivity in the elementary basis for symmetric functions. If P is a partially ordered 
set, then the incomparability graph of P is the graph inc(P) whose vertices are the 
elements of P and whose edges are the pairs of vertices which are incomparable in 
P. A poset is called (r + s)-free if no induced subposet is isomorphic to the direct 
sum of a chain (totally ordered set) with r elements and a chain with s elements. 


Conjecture 5.1. (Stanley-Stembridge Conjecture [118, 121]) Jf G=inc(P) for 
some (3 + 1)-free poset P, then X g(x) is e-positive. 


For example, the poset P in Fig. 10 is (3 + 1)-free. The e-expansion for the chro- 
matic symmetric function corresponding to its incomparability graph is Xinc(p) = 
4e3; + 8e4. 

The incomparability graph for a (3 + 1)-free poset is an example of a claw-free 
graph. A claw-free graph is a graph which does not contain the star graph $3 (depicted 
in Fig. 11) as asubgraph. However, not all claw-free graphs are e-positive; Dahlberg, 
Foley and van Willigenburg [27] provide a family of claw-free graphs which are not 
e-positive. 

Gasharov [42] proved that the incomparability graph of a (3 + 1)-free poset is 
Schur-positive. Since the elementary symmetric functions are Schur-positive, Schur 


a Cc d a————¢ 
<7 a 
P inc(P) 


Fig. 10 A poset P and its incomparability graph 
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Fig. 11 Star graph $3 on a b 
four vertices \ 

c d 
Fig. 12 A labeling of the —1)) 


star graph $3 on four vertices | a 
4 3 


positivity would follow immediately from e-positivity. Guay-Paquet [48] proved that 
the chromatic symmetric function of a (3 + 1)-free poset is a convex combination 
of chromatic symmetric functions of posets which are both (3 + 1)-free and (2 + 
2)-free. This reduces the e-positivity conjecture to a subclass of posets with more 
structure than posets which are (3 + 1)-free. A natural unit interval order is a poset 
P on the set [m] := {1,2,...,} obtained from a certain type of intervals on the 
real line as follows. Let {[a,, aj + 1], [a2, a2 + 1], ..., [Gy, Qn + IJ} be a collection 
of closed intervals of length one such that a; < aj, for! <i <n—1.Seti <p j 
if a; + 1 < a;. The resulting partially ordered set will always be (3 + 1)-free and 
(2 + 2)-free, and in fact, every poset that is both (3 + 1)-free and (2 + 2)-free is a 
unit interval order [112]. 

Shareshian and Wachs recently proposed a new approach to the Stanley— 
Stembridge e-positivity conjecture in the form of a refinement of Stanley’s chromatic 
symmetric functions. This refinement behaves nicely with respect to unit interval 
orders. 


Definition 5.2. ({115]) Let G = (V, E) be a graph whose vertex set V is a finite 
subset of P. The chromatic quasisymmetric function of G is 


Xg(x,t) = > peels) 6 
KEC(G) 


where 
asc(kK) = |{{i, j} € E |i < j and K@) < K(J)}I. 


Notice that the chromatic quasisymmetric function X g(x, t) depends not only on 


the isomorphism class of the graph G but also on the labeling of the vertices of G. 
Let G be the star graph on four vertices labeled as shown in Fig. 12. Then 


XG (x,t) = M3, + Mj21 + May, + Mii + t(M121 + 2Mo11 + 3M1111)+ 
+17(Mi21 + 2Moi1 + Mio + 5M) +O (Mya1 + 2M 12 + Mary + M3 + 6Mi141)+ 


+14(My21 + 2M112 + 5M) + OM 21 + Mira + 3M) + 8M). 
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The chromatic quasisymmetric function reduces to the chromatic symmetric function 
by setting t = 1; that is Xg(x, 1) = XG(x). 

Let w be the involution map on QSym which sends Fs to Ft,—1)\s. The image 
of Xg(x,t) under w has a natural positive expansion into the fundamental basis 
for quasisymmetric functions [115]. Shareshian and Wachs [114] further conjecture 
that when G is the incomparability graph of a natural unit interval order, this image 
corresponds to a sum of Frobenius characteristics associated to certain Hessenberg 
varieties. This conjecture was proved by Brosnan and Chow [22] and, through a 
different approach, by Guay-Paquet [49], providing an alternate proof of Schur pos- 
itivity. 

Theorem 5.3. ({115]) If G is the incomparability graph of a natural unit interval 
order, then X g(x, t) is symmetric in the x-variables. 


Not every graph whose chromatic quasisymmetric function is symmetric is an 
incomparability graph of a natural unit interval order. One interesting open question 
is to classify which graphs admit a symmetric chromatic quasisymmetric function. 

Several extensions of chromatic quasisymmetric functions have recently emerged, 
demonstrating the many different areas this research impacts. Ellzey extends this 
paradigm to directed graphs [35]. Haglund and Wilson express the integral form Mac- 
donald polynomials as weighted sums of chromatic quasisymmetric functions [53]. 
Clearman, Hyatt, Shelton, and Skandera interpret the chromatic quasisymmetric 
functions in terms of Hecke algebra traces [26], while Alexandersson and Panova 
connect the chromatic quasisymmetric functions to LLT polynomials [2]. 


5.2. Quasisymmetric Expansions of Symmetric Functions 


As quasisymmetric functions become more ubiquitous, many natural expansions of 
symmetric functions into quasisymmetric functions (particularly into the fundamen- 
tal quasisymmetric functions) are appearing. It is natural to try to use this structure 
to answer classical questions about symmetric functions such as Schur positivity. 
Egge, Loehr, and Warrington [33] recently introduced a method to convert the qua- 
sisymmetric expansion of a symmetric function into the Schur function expansion, 
providing a new approach to questions of Schur positivity. 

We need several definitions in order to describe the “modified inverse Kostka 
matrix” and some interesting applications of this paradigm. A rim-hook is a set of 
contiguous cells in a partition diagram such that each diagonal contains at most one 
cell. A special rim-hook tableau is a decomposition of a partition diagram into rim- 
hooks such that each rim-hook contains at least one cell in the leftmost column of the 
diagram. Egecioglu and Remmel [36] use special rim-hook tableaux in their formula 
for the inverse Kostka matrix, which 1s the transition matrix from the monomial basis 
for symmetric functions to the Schur functions. 

The sign of a special rim-hook is (—1)’~!, where r is the number of rows spanned 
by the rim-hook. The sign of a special rim-hook tableau is the product of the signs of 
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its rim-hooks. A special rim-hook tableau is said to be flat if each rim-hook contains 
exactly one cell in the leftmost column of the partition diagram. 


Theorem 5.4. ((33]) Let F be a field, and let f be a symmetric function given by 
its expansion into the fundamental quasisymmetric functions so that 


f= y velo: 


aren 


Then the coefficients x) in the Schur function expansion f = 2 X)S) are given by 
AFn 


x, = = YoK; (a, ), 


aren 


where K*(a, A) is the sum of the signs of all flat special rim-hook tableaux of partition 
shape X and content a. 


Theorem 5.4 provides a potential alternative approach to proving that Macdonald 
polynomials expand positively into the Schur functions. In particular, recall that 
Theorem 3.2 describes a formula for expanding Macdonald polynomials into the 
fundamental quasisymmetric functions. Combining this formula with Theorem 5.4 
implies that the coefficient of s) in the Schur function expansion of A, is given by 


a Ke (a, ) > giv G.w pmaiB.n) 


atn Be{G,,|Des(3-!)=a} 


The following example is similar to that appearing in [33]. If w = (3, 1) and 
A = (2, 2), then there exists a flat special rim-hook tableau for a = (2, 2) and a flat 
special rim-hook tableau a = (1, 3). These correspond to 


Ki ((2, 2), (2, 2)) = +1 and K7((1, 3), (2,2)) = -1 


respectively. The permutations w whose inverse descent sets Des(w~!) are {2} are 
3412, 3142, 3124, 1324, and 1342. Computing the inv and maj for the fillings of 
(3, 1) with these permutations as reading words produces 2q7 + qt + t + q. Simi- 
larly, the permutations whose inverse descent sets are {1} are 2341, 2314, and 2134. 
Their inv and maj (for fillings of (3, 1)) produce —q* — q — t. Putting this together, 
the coefficient of 59 in H3 is 


2g? tqt+t+q-@+qtn=q tat. 
Notice that negative terms do appear in the K;(a, A). This means that in order 


to apply this technique to the Schur positivity of Macdonald polynomials problem, 
one must find involutions to cancel out the negative terms. 
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A further application of this transition matrix from the fundamental quasisymmet- 
ric functions to Schur functions is to the Foulkes Plethysm Conjecture [38], which 
states that s,[5,] — 5m[5,] (where the brackets denote a certain type of substitution 
called plethysm) is Schur-positive. Loehr and Warrington [83] provide a formula for 
the expansion of s,,[s,] into fundamental quasisymmetric functions using a novel 
interpretation of the “reading word” of a matrix. The modified inverse Kostka matrix 
could then be used to determine the Schur function expansion of s,[5n] — Sm[Spl, 
again with the caveat that involutions are needed to cancel out the negative terms. 

Garsia and Remmel recently found a further extension of the Egge, Loehr, War- 
rington result. They proved that each fundamental appearing in the fundamental 
expansion of a symmetric function can be replaced by the Schur function indexed 
by the same composition. Since every such Schur function is either 0 or +s) for 
some partition A, this expansion can be simplified to a signed sum of Schur functions 
indexed by partitions. 


Theorem 5.5. ((40]) Let f be asymmetric function which is homogeneous of degree 
n and expands into the fundamental basis for quasisymmetric functions as follows: 


f= a Ay Fy. 


Then 


f= > AgSe- 


afen 


Theorem 5.5 already has a number of important consequences. Garsia and Remmel 
used this approach to formulate a conjecture regarding the modified Hall—Littlewood 
polynomials. Leven applied this method to prove an extension of the Shuffle Con- 
jecture for the cases m = 2 and n = 2 [80]. Qiu and Remmel considered the cases 
of this “Rational Shuffle Conjecture” where m or n equals 3 [102]. 


5.3 Slide Polynomials and the Quasi-key Basis 


Schubert polynomials are an important class of polynomials, first introduced by 
Lascoux and Schiutzenberger [77] to provide a new method for computing intersec- 
tion numbers in the cohomology ring of the complete flag variety. Several different 
combinatorial formulas for Schubert polynomials have been discovered since their 
original introduction as divided difference operators, including but not limited to 
reduced decompositions [21, 37] and RC-graphs [15]. Despite the numerous ways 
to construct Schubert polynomials, it remains an open problem to provide a com- 
binatorial formula for the expansion of a product of Schubert polynomials into the 
Schubert basis. 
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Assaf and Searles [5] further the study of Schubert polynomials with the intro- 
duction of two new families of polynomials, both of which positively refine the 
Schubert polynomials. These new families, called the monomial slide polynomials 
and the fundamental slide polynomials, exhibit positive structure constants (mean- 
ing the coefficients appearing in their products are always positive), whereas the key 
polynomials (another family of polynomials refining the Schubert polynomials [28, 
78, 105]) have signed structure constants. Although the slide polynomials have many 
interesting applications (to Schubert polynomials and other objects of study in alge- 
braic combinatorics), this article focuses on their connections to quasisymmetric 
functions. 

Remove the zeros from a weak composition y to obtain a (strong) composition 
called the flattening of 7, denoted b(y). The monomial slide polynomial JN, is then 
defined by 

MN, (41, 42.6.2, %n) = > ea + xen, 


b> 
b(d)=b(y) 


where 6 > + if 6 dominates 7; that is 6; + 6. +--- +6; >y+72+---+7 forall 
1 <i <n. The related fundamental slide polynomial §, is defined by 


= 51 62 5n 
oy = ) XX KL" 


b=y 
b(5) refines b(y) 


For example, 


2 2 2 2,2 2 2 3 
§1032(*1, X2,%3,%4) = Hea + Hie + titeKe + XY XQAZXG + ALA AZBAY + XAZAZAG 
= Myros2 + Myi22 + Mi212 + Misi. 


Each of these families {9t,}, and {%,}, (indexed by weak compositions of k) of 
polynomials is a Z-basis for polynomials of degree k in n variables. 

Assaf and Searles [4] also introduce a related basis, called the guasi-key polyno- 
mials Q.,, for the polynomial ring which is analogous to the key polynomials. These 
polynomials are positive sums of fundamental slide polynomials and in fact stabi- 
lize to the quasisymmetric Schur functions as zeros are prepended to their indexing 
compositions. (Prepending m zeros to the composition 7 is denoted by 0” x 7.) 


Theorem 5.6. ([4]) For any weak composition y, we have 


Jim Doe = Si 

Each Schubert polynomial can be written as a positive sum of fundamental slide 
polynomials using a new object called a quasi-Yamanouchi pipe dream. While this 
definition takes us too far from our current topic, we do take the time to describe a 
closely related construction involving the fundamental expansion of Schur functions. 
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Recall that the Schur functions decompose into a positive sum of the fundamental 
quasisymmetric functions (see Eq. 1.1); this formula can be computed by finding 
the descent sets of all standard Young tableaux of a given shape. However, when 
the number of variables is less than the number of descents, the corresponding fun- 
damental equals 0. Assaf and Searles [5] introduce a class of semi-standard Young 
tableaux, called quasi- Yamanouchi tableaux, which dictate precisely which funda- 
mentals appear in the decomposition with nonzero coefficient when the variables are 
restricted. 


Definition 5.7. ((5]) A semi-standard Young tableau is said to be quasi- Yamanouchi 
if for alli > 1, the leftmost occurrence of i lies weakly left of some appearance of 
i — 1. Let QYT,,(A) denote the set of quasi-Yamanouchi tableaux of shape 4 whose 
entries are in [n]. 


The weight of a quasi- Yamanouchi tableau T is given by wt (T) = []; x;", where 
m; is the number of times the entry i appears in T. 


Theorem 5.8. ((5]) The Schur polynomial sy(x,,..., Xn) is given by 


S\(X1,---,Xn) = SS Fut r)(X1, «++ 5 Xn). 
TEQYT,(d) 


For example, the three quasi- Yamanouchi tableaux of shape \ = (4, 2) and entries 
in {1, 2} are 


2|2 2:|.2 2|2 
1 1jijl 1/1], 1)2 1) 1|2)2 


and therefore, the Schur expansion into fundamentals is 
$42(%1, X2) = Fa4.o(%1, X2) + F3,3(%1, x2) + Fo.4(r1, X2). 


Note that all the terms appearing on the right-hand side are nonzero, and there is 
no need to calculate the descent sets for all nine standard Young tableaux of shape 
(4, 2). This is important in the study of Schubert polynomials because although 
certain classes of Schubert polynomials are equal to Schur functions, the number 
of variables appearing varies based on the indexing permutation. For example, the 
Schubert polynomial indexed by the permutation 213 (written in one-line notation) 
is equal to the Schur function sj (x) = x; while the Schubert polynomial indexed by 
the permutation 132 (written in one-line notation) is 51 (x1, x2) = x; + X2. A thor- 
ough understanding of precisely the nonzero terms appearing in the quasisymmetric 
expansion is therefore crucial to the quest of proving a combinatorial formula for 
Schubert multiplication. 

This connection to Schubert multiplication (a long-standing open problem in alge- 
braic combinatorics) exemplifies the utility of quasisymmetric functions. Quasisym- 
metric functions appear in a number of other important problems which have helped 
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to shape the study of algebraic combinatorics including Schur positivity of Mac- 
donald polynomials, the Foulkes plethysm conjecture, and the Stanley-Stembridge 
conjecture. We hope the reader comes away from this article with a deeper appreci- 
ation for the beauty and utility of quasisymmetric functions and a desire to further 
explore this exciting and far-reaching avenue of research. 
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On Standard Young Tableaux ®) 
of Bounded Height siete 


M. J. Mishna 


Abstract We survey some recent works on standard Young tableaux of bounded 
height. We focus on consequences resulting from numerous bijections to lattice 
walks in Weyl chambers. 


1 Introduction 


Standard Young tableaux are a classic object of mathematics, appearing in prob- 
lems from representation theory to bijective combinatorics. Lattice walks restricted 
to cones are similarly a fundamental family, and they encode a wide variety of com- 
binatorial structures from formal languages to queues. Standard Young tableaux of 
bounded height are in bijection with several different straightforward classes of lat- 
tice walks. This connection not only elucidates several sources of ubiquity on both 
accounts, but facilitates exact and asymptotic enumeration, as well as parameter 
analysis. This survey describes the cross developments over the past 30 years, and 
highlights some open problems. For more background on the tableaux, we recom- 
mend the surveys [1, 30] and particularly the article of Stanley [34]. 

We begin by fixing our notation and conventions. Let A = (Aj, A2,..., A;) be a 
partition of n into ¢ parts. We write A n and £(A) = ¢ in this case. The Ferrers 
diagram of shape 2 is a representation of \ comprised of boxes indexed by pairs 
{G, jf): 1<i<t;1<j <j}. Such a diagram is of size n and height t. A key 
parameter in our study is the number of columns of odd length. 

A standard Young tableau of size n is a filling of a Ferrers diagram using precisely 
the integers | ton. The entries strictly increase to the right along each row and strictly 
increase down each column. A tableau is semi-standard if the entries weakly increase 
along each row and strictly increase down each column. In this work, we are interested 
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in the number of standard Young tableaux of a given size with the height is bounded 
by a fixed value. 
1.1 Enumeration Formulas 


Rather classically, the number of standard Young tableaux of shape 2 is denoted f* 
and is given by the hook-length formula: 


where he = 4; + card{j: A; >i}-i-—jtl. (1) 
The following formulation is due to MacMahon: 


1 
rd 
= tot Aptaet (5) 
f : Oi — t+ AY) <i jea 


The number of standard Young tableaux of height at most k is thus the sum 


nin) = D> fr. (2) 
Aen 
L(A) <k 


The first enumerative formulas for Young tableaux where the height is an explicit 
consideration appear in the 1960s, when Gordon and Houten studied k-rowed plane 
partitions whose nonzero parts strictly decrease along rows and columns, in addition 
to some related variants. In their series of Notes on Plane Partitions [20, 21], they 
give some formulas for the generating functions in terms of infinite products and 
determinants. Regev [29] first determined exact expressions for y2(”) and y3(n). The 
formulas are equivalent to the following. Here C;, denotes the kth Catalan number: 


[n/2] 
n nN 
y(n) = oe y3(n) = )> (Co. (3) 


k=0 


The numbers y3(m) are also known as Motzkin numbers. Figure | illustrates the 
nine standard Young tableaux of size four and of height at most three. Around the 
same time, Gessel [16] found an expression for y4(7), and Gouyou-Beauchamps [23] 
found the following expressions for y4(n) and ys5(n): 
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Fig. 1 All standard Young 1]213 1)2]4 113]4 
tableaux of size 4 and height HS) fa 3 2 
at most 3 Tal fala 4 
1}2 1/3 
3|4 2\4 2 = 2 
4 4 3 


y4(2n) = CiCn y4(2n a 1) = CrCn+1 


pals 3!n!(2i + 2)! (4) 


y5(n) = 2 (n — 2i)WIG + DIG + 2!E4+ 3) 


No comparable expression for yg(”) has appeared in the literature. The presence of 
binomials in general, and Catalan numbers in particular, is a strong hint that these 
tableaux are related to well understood combinatorial classes. 


1.2. The Exponential Generating Function 
We study y;(7) via Y;(t), the exponential generating function for y; (7): 


nt) = ey. (5) 


The formulas depends on the parity of the height. The formula for Y2; (t) was obtained 
by Gordon [19] by reducing a Pfaffian of Gordon and Houten [20], and Gessel [16] 
found the formula for odd heights. They are both expressed in terms of the hyperbolic 
Bessel function of the first kind of order j 


co pent 
b; =1,2t)= oe 
J J 2 ni(n+ j)! 
The formulas are: 
Yo, (t) = det (bi; + Biaj-t) vot 428 (6) 
Yorpi(t) = e’ det (bij = Dieses 1) 


For example, Y(t) = bo + b; and Y4(t) = be + bob, + bob3 — 2b, b2 — bs _ bt 
+ b,b3. These expressions grow very fast as polynomials in b;; however, they are 
amenable to some further study including asymptotic analysis of the coefficients. 
They also imply that y,(”) can be expressed by binomial sums, although an expres- 
sion that one could compute may be too complex to be of any use. 

One important consequence of these formulas is that they resolved a question that 
Stanley [35] had asked almost 10 years before these formulas appeared about the 
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nature of the function Y;(t). Specifically, he asked if Y;(t) is D-finite, that is, does 
it satisfy a differential equation with polynomial coefficients. Gouyou-Beauchamps 
showed that Y4(t) is D-finite from his formula, and Gessel [16] proved this result for 
general k since the expressions are a polynomial combination of Bessel functions 
(possibly times an exponential). Bessel functions are D-finite, so the result follows 
from closure properties. 


1.3 Schur Functions 


To prove these formulas, Gessel started with another important formula for standard 
Young tableaux. Schur functions can be described via a summation over the set of 
all semi-standard Young tableaux of shape A: 


= content(T) __ ti hh th 
i= yo aS. Fae. (8) 


TeSSYT(A) TeSSYT(\) 


The exponents describe the content of the tableau: ft; is the number of occurrences 
of i in T. Thus, the coefficient of the monomial x, ...x, in this expression is the 
number of standard Young tableaux of shape A. We deduce the formula 


y(n) = > [xX1X2...Xy]S) = [x1 x2...Xy] > Sy. 


AEn Aen 
€(\) sk €(A) sk 


This kind of coefficient extraction in symmetric functions can be framed as a 
homomorphism. This was done by Gessel in his PhD thesis (Theorem 3.5) and also 
by Jackson and Goulden [22, Lemma 4.2.5]. In the case of the homogeneous complete 
symmetric function h,, it is easy to see that [xjx2...X,Jhg = 1,=,~. One would like 
to apply this to the Jacobi-Trudi identity, which is an expression for a Schur function 
in terms the homogeneous complete symmetric functions: 


5S) = det (hx ot F 

The truth is slightly more complicated. Indeed Gordon and Houten did much of 
the heavy lifting for this problem and had expressed the number of semi-standard 
tableaux as a determinant of homogenous complete symmetric functions. Gessel 
extracted the generating function for standard Young tableaux and derived the for- 
mulas in Eq. (6). 
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1.4 The Robinson-Schensted Correspondence 


The following identity is incredibly evocative to combinatorialists: 


ary =a! (9) 


AFn 


The bijective correspondence between pairs of standard Young tableaux of the same 
shape and permutations was described by Robinson in the 1930s and also by Schen- 
sted in the 1960s. Below we describe the Schensted algorithm which builds a pair 
of tableaux by parsing the permutation and incrementally building two tableaux. We 
refer readers to Sagan’s book [31] for additional details. 

An algorithmic description of the bijection begins with a permutation o € S,, and 
a pair of empty tableaux, denoted (Po, Qo). Fori from | ton, we create P; by adding 
a box with entry o(i) to P;_; via a special insertion algorithm. For each i we add 
precisely one box. The position is noted and a box with entry i in the same location is 
added to Q;_;. The sequence of Q tableaux records the history of the box additions. 
The pair (P,,, Q,) is returned. 

The row insertion takes as input a possibly incomplete standard Young tableau and 
adds an integer m not already in the tableau. The process acts incrementally along 
each row. If m is bigger than all of the elements in the row under consideration, it 
is placed at the end of it. If not, it finds its natural place and bumps the larger value. 
The bumped value is inserted in the tableau formed by the lower rows by the same 
process. 


Example I. Let o be the involution o = 72961041835. If the Schensted algo- 
rithm is applied, the penultimate step returns the following two tableaux: 


p, = 2/318] 9, =[1 [315] 
2[4|10 2(4[8 
6[9 6/9 
7 7 


The final step is to insert 5 into Py. The 5 fits between the 3 and 8 in the first row, 
hence it bumps 8 to the next row which then bumps 10 which settles at the end of row 
three. The position of the new square is (3, 3), and the algorithm finishes by adding 
this square to Qo, with an entry value of 10. 

The end result is 


WwW 
Nn 


Pio = Qi0 = 


A 
oo 


SYD, M] rR 
\o 
— 
=] 
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The involution has two fixed points (2 and 8) and the resulting tableaux each have 
two columns of odd length (specifically, length 3). 


The example illustrates some properties which are true in general. The length of 
the longest increasing subsequence of o is equal to the length of the first row of 
the tableaux. If o is an involution, then the number of fixed points of o equals the 
number of columns of odd length in \. That standard Young tableaux of size n are 
equinumerous with involutions was known to Frobenius and Schur. 

Viennot [38] described a very beautiful geometric construction using shadow 
lines to give a more intuitive illustration of this slightly mechanical bijection. It 
should be better incorporated into the bijections we encounter in later sections. A 
second geometric construction is given by Fomin using growth diagrams [14]. The 
construction can be mined for additional information. 


1.5 Plan of the Article 


The first formulas found Y;(t) were recognized to resemble generating functions for 
walks in Weyl chambers deduced by Grabiner and Magyar [25, Section 6.2]. We 
examine the relevant developments made in the early 1990s [15, 39, 40, 42, 43] in 
Sect. 2. 

Bijective proofs of some of these connections are more recent [10, 26, 33], 
although the work of Gouyou-Beauchamps dates back to the late 1980s. All of these 
authors’ proofs pass through secondary objects, such as coloured Motzkin paths, or 
matchings. We consider these in Sect. 2.6. 

One of the most common strategies for enumerating lattice walks restricted to 
cones with symmetry in the set of allowable steps involves a sub-series extraction 
from a rational function. Remarkably, many combinatorial classes with transcen- 
dental D-finite generating functions share this property. It is an open problem to 
answer under which conditions this might be universally true. In part, it is a useful 
formulation since in ideal cases we can answer questions about the order and com- 
plexity of the recurrences and also (re-) derive asymptotic formulas. We describe 
such extractions, and the resulting tableaux generating functions in Sect. 3. 

From the Robinson—Schensted correspondence, we see that the combinatorial 
cousin to the standard Young tableau of bounded height is the permutation with 
bounded longest increasing subsequence. Many of the techniques described here can 
also enumerate these classes. For example, Gessel determined expressions for the 
generating functions, and they also have lattice walk interpretations. We summarize 
results in Sect. 4. 

We conclude with some natural generalizations and open problems. 
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2 Lattice Walk Models 


There are no fewer than five different lattice walk classes that are equinumerous 
with standard Young tableaux of bounded height. Most of them are defined using 
d-dimensional Wey] chambers” of type C, denoted by Wc(d) 


We(d) = {(1, X2,...,Xg) 2X1 = XQ S++ [| Xg = OF. 


Let {e),..., eg} denote the standard basis of R@. A lattice walk model is defined 
by a set of allowable steps and a region which confines each walk. A walk is a 
sequence of steps. For example, the set 


S = {te :1<i<d} 


defines the d dimensional simple step set. This is considered in several different 
bounding cones. 

Gessel and Zeilberger [17] considered general walks in Weyl chambers and 
demonstrated an enumeration strategy for models where the stepset possesses a cer- 
tain kind of symmetry, and avoid jumping over boundaries. Such walks have come 
be called walks reflectable. The generating function for reflectable walk models with 
specified starting and endpoints (excursions) is written as a coefficient extraction 
from a signed sum of unrestricted walks. This idea, which appears frequently in lat- 
tice walk enumeration, is a version of the reflection principle. The expressions were 
made explicit for Wc(d) by Grabiner and Magyar for the simple step set, which 
happens to be reflectable. 


Theorem 1 (Grabiner and Magyar [25]). For fixed X, 1 € Wc(d), the exponential 
generating function O),,(t) of the simple walks in Wc(d) from X to p, counted by 
their lengths, satisfies 


Oy p(t) = det Dis, _ Duds) eg i . 


This is immediately reminiscent of the generating function formulas in Eq. (6). 
Several authors have made the connection, in particular, Gessel, Weinstein and 
Wilf [15], Zeilberger [43], Xin [42], Euetal. [32, 33], Burrill et al. [10] and Courtiel et 
al. [12]. In almost every case, there is a natural parameter which is equidistributed 
with number of odd columns in the tableaux. We describe these classes next. 


For convenience we define the chambers using non-strict inequalities, our bijective statements can 
equivalently be given under strict inequalities, upon applying the coordinate shift x; = x; + d+ 
1-i. 
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2.1 Ballot Walks 


We can build an integer sequence from the entries of a standard Young tableaux. 
We define the associated lattice word w = (wi, W2,..., Wn) by setting w; = / if the 
entry i is in the jth row. This word has the property that for any prefix the number of 
occurrences of £ is greater than or equal to the number of occurrences of £ + 1 since 
the columns of the associated tableau are strictly decreasing. These are also known 
as generalized Ballot sequences. 

We associate a step naturally to each letter: 


WwW, 41, Wi > ej — ej-1 for (2<i<k-—1l1), Wk —ekx-}- 
Restricting walks to the first orthant is equivalent to the ballot condition. That is, the 
ballot word of a tableaux of height at most k gives a natural encoding as a lattice 
walk in the cone RS with steps from the following stepset: 


B= {ey —G} U1 ea 2 SS kT), 


Figure 2 has an example. 


1/3/5 

2|4/8 

6 =Teloho 

7 
12345 6789 10 
ee 2 9 § 141 & 3 5 


w= (by, be, bi, bo, by, b3, ba, ba, b3, b3) 


Ji 


p=l2345678910 


a= (2, , 3 ? , ? ? 9 ? ? ) 


Fig. 2 Representatives of classes of objects in bijection with standard Young tableaux of bounded 
height, in particular their image of 1, a standard Young tableau of height 3 with two odd columns; 
a is an involution with maximal increasing sequence of length 3; w is a ballot walk; jz is an open 
arc diagram for a partial matching; 6 is an oscillating tableau ending on a row shape 
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2.2 Lazy Walks 


To define this step set, we set 0 to be the zero step (whence the label “lazy”). 
Zeilberger [43] noted (although, he attributes the proof to Xin without citation) that 
the number of excursions in Wc (k) starting and ending at the origin using the stepset 
L, given by 

L= {e;,-e;: 1 <i <k}U {0} 


iS y2x41(). A small computation suggests that the distribution of the zero steps 
matches the distribution of odd columns in the Young tableaux. He remarks that it 
would be interesting to find a (bijective) proof of this result. 


2.3 Generalized Motzkin Paths 


In Zeilberger’s lazy walks, the k = | case encodes the classic Motzkin walks, con- 
sistent with the longstanding observation that y3 (7) is the number of Motzkin words. 
One could view the higher dimension lazy walks as a generalization of Motzkin 
words, and this notion was first formalized by Eu [32], and subsequently by Eu, Fu, 
Hou, and Hsu [33]. They add a counter component and describe an explicit bijection 
between the Motzkin paths of length n and the standard Young tableaux of size n 
with at most three rows. 

To prove this, they considered the lazy lattice walks and then algorithmically 
mapped the steps in. The odd and even cases reconcile as follows. 


Theorem 2 (Eu, Fu, Hou, and Hsu 2013 [33, Theorem 1.1]). Consider the lattice 
model defined by the step set 


M = {ei} U {e1 + e2, e1 — en} U {e1 — ee; + ei41,€1 $e; — i411 2 <1 <k} 


confined to Zz The number of walks of length n from the origin to the point 
(n, 0, ..., 0) staying within the nonnegative octant equals the number of n-cell SYTs 
with at most 2k + | rows. 

If, additionally, the e, steps are confined to the hyperplane spanned by the vectors 
{e1,..., ex}, then the number of paths equals the number of n-cell standard Young 
tableaux with at most 2k rows. 

The number of e, steps is equidistributed with the odd column statistic of standard 
Young tableaux. 
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® 


Fig. 3 The first few levels of Young’s lattice of Ferrers diagrams 


2.4 Oscillating Tableaux and Arc Diagrams 


The set of Ferrers diagrams ordered by diagram inclusion? is called Young’s lattice. 


Figure 3 depicts the first few levels of its Hasse diagram. We consider a sequence 
of Ferrers diagrams as a walk on this lattice. We consider three variants defined 
by restrictions on moving up or down in the lattice (or not at all). The length of a 
sequence is the number of elements, minus one. (It is the number of steps in the 
corresponding walk.) 

An oscillating tableau is simply a sequence of Ferrers diagrams such that at every 
stage a box is either added or deleted. They were popularized by their use in interpre- 
tations for representations of the symplectic group [36]. If no diagram in the sequence 
is of height k + 1, we say that the tableau has its height bounded by k. We recall two 
related families, namely vacillating tableaux, and hesitating tableaux. The vacillat- 
ing tableaux are even length sequences of Ferrers diagrams, written (A, ..., A°”) 
where consecutive elements in the sequence are either the same or differ by one 
square, under the restriction that \?) > \?+) and ACD < \C'+2, The hesitat- 
ing tableaux are even length sequences of Ferrers diagrams, written (A, ..., A°”) 
where consecutive differences of elements in the sequence are either the same or 
differ by one square, under the following restrictions: 


e if X27) = VCD), then APY) < P42) (do nothing; add a box) 
e if KX?) > NCD, then \A+) = \P42) (remove a box; do nothing) 
e if M2) < NA+) then \2+D s \P4+2) (add a box; remove a box). 


Figure 4 shows examples from each of these classes. 

Chen, Deng, Du, Stanley, and Yan [11] described nontrivial bijections between 
sequences of Ferrers diagrams and several combinatorial families encoded in arc 
diagrams. Arc diagrams are labelled graphs under some degree and embedding 
restraints. They can be used to represent a variety of combinatorial classes, such 
as matchings and partitions. Figure 5 illustrates how to encode a set partition as an 
arc diagram. 


3Recall \ < je means that Aj < jy; for all i. 
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(2,0, ? ? rf ? ? d ) 
(2, 2 ? ? ? ? ? ? 7 > ? ) 


Fig. 4 From top to bottom: a vacillating tableau of length 10; a hesitating tableau of length 8; an 
oscillating tableau of length 11. In each case, the height is bounded by 2. From [10] 


“er e 
1234567 8 910 


Fig.5 An arc diagram representation of the set partition 7 = {1, 2, 7, 10}, {3, 8}, {4, 6}, {5S}, {9}. It 
is both 3-noncrossing and 3-nonnesting. However, the subdiagram induced by {2, 3, 7, 8, 10} is an 
enhanced 3-crossing. Similarly, the subdiagram induced by {3, 4, 5, 6, 8} is an enhanced 3-nesting 


They generalize the Schensted algorithm, in some sense, by describing how to 
parse arc diagrams, with each step defining a tableau insertion or deletion. The result 
is bijection ¢@ from partitions to vacillating tableau. It is robust and can be adapted 
to other arc diagram classes. A key feature is sub-pattern avoidance properties in the 
arc diagrams are mapped to height restrictions on the tableaux. This echoes a key 
feature of the Schensted algorithm. 

An arc diagram is noncrossing if no two arcs intersect. Noncrossing set partitions 
are counted by Catalan numbers. In addition to appearing in combinatorics, these 
diagrams arise in algebra, physics, and free probability. The notion of a crossing 
is generalized to a k-crossing, which denotes a set of k arcs that each mutually 
cross. Similarly, a k-nesting refers to k arcs which mutually nest into a rainbow 
figure. More formally, let us consider a set of k distinct arcs: (i), j1),.--, Gk, Jx)- 
They form a k-crossing if i) < ig <--+ <ip < ji < jo <--++ < jg, anda k-nesting 
if ip <ig <-++ <ip < jp <-++-+ < jo < jy. A Slightly relaxed notion is sometimes 
appropriate. Enhanced k-nestings and k-crossings permit the middle inequality to 
be an equality. We are interested in the classes of diagrams which avoid such sub- 
diagrams and consider k-noncrossing diagrams (they contain no k-crossing) and 
k-nonnesting diagrams (which contain no k-nesting). Figure 5 gives examples. 

Theorem 3.2 of Chen et al. [11] proves that given a partition 7 of {1,...n}, and 
the vacillating tableau (7) = (@ = A, AY, ..., AC = B), the size of the largest 
crossing of ¢ is the largest height of any );, and the size of the largest nesting is the 
maximum number of columns. 

Thus, by a simple transposition one deduces the corollary comprised of the surpris- 
ingly nontrivial fact that k-noncrossing set partitions of {1, 2, ..., m} are in bijection 
with k-nonnesting set partitions of {1, 2, ...,}. We leave it to the reader to see why 
a simple swap of sub-diagrams is not a bijection. 

The bijection ¢ supports many generalizations. A diagram is said to be open if, 
in addition to closed edges, there are also half edges. The notion of crossing extends 
naturally, given a fixed convention of how the open edges lie. The diagram jz in Fig. 2 
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demonstrates an incomplete, or open, matching. Remark there are not three edges 
that mutually cross, and hence, this diagram is also 3-noncrossing. An open diagram 
corresponds to a sequence that does not necessarily end at an empty diagram— 
rather it ends in a row shape, i.e. a partition with a single part: (m). The bijection 
@ is well defined in this case, although Chen et al. did not discuss how to interpret 
this, although Burrill, Melczer and Mishna [9]. Xin [42] consider walks with any 
endpoint and found generating functions for palindromic sequences. 

In a sequence of Ferrers diagrams of height at most k, each diagram can be coded 
by a k-dimensional vector where each successive entry is weakly decreasing. Thus, 
a sequence is an encoding of a walk in Wc(d). Thus, we can view the bijection ¢ as 
a nontrivial map between arc diagrams and walks. To end in a row shape is the same 
as ending on the first axis. 

This brings us back to our topic at hand, standard Young tableaux. Which arc 
diagrams are associated with a standard Young tableau of bounded height? The most 
plain interpretation is to view a standard Young tableau as a sequence of Ferrers 
diagrams—it starts at the empty diagram and a single cell is added at each step, with 
the position indicated by the entry in the tableaux. We could consider the arc diagram 
image of this sequence, but we can do better. 

Recall the Robinson—Schensted correspondence is a bijection between pairs of 
standard Young tableaux and permutations. By restricting the map to pairs of identical 
tableaux, it becomes a bijection between standard Young tableaux and involutions. 
Involutions have a very natural arc diagram representation! In this correspondence, a 
tableau with a fixed number of odd columns is mapped to an involution with precisely 
that many fixed points. An involution is a partial matching, and they are in bijection 
with oscillating tableaux. 

These pieces lead to the following result, a map between standard Young tableaux 
and a class of walks that end on a boundary. 


Theorem 3 (Krattenthaler [26, Theorem 4]; Burrill, Courtiel, Fusy, Melczer and 
Mishna [10, Theorem 1]; Okada [28, Theorem 1.2]). Forn, k => 1, there is an explicit 
bijection between the standard Young tableaux of size n with height at most k and 
with m odd columns, and the simple walks of length n staying in Wc(k), starting 
from the origin and ending at the point m ey. 


We have outlined one possible argument, but in fact three very different proofs 
have been discovered. Above is the argument of Burrill, Courtiel, Fusy, Melczer 
and Mishna [10]. Krattenthaler [26] determines a different bijection using growth 
diagrams as an intermediary object. It is direct and gives several generalizations. It is 
self-contained in that it does not exploit the 7 map. Okada [28] uses a representation 
theoretic argument starting from Pieri rules. He also has generalizations to semi- 
standard tableaux. Gouyou-Beauchamps proved the k = 4 case. 

One consequence of a lattice walk bijection is a generating function expression. 
Applying Grabiner and Magyar’s formula gives the following 


k-1 2k—1—2u 


Y(t) = D0(-D* 5 Gp) det(bj_j — dedi jo<i<k—-Ligus'sj<k-1- 


u=0 =u 
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Remarkably, the infinite sum which arises from direct application of Grabiner and 
Magyar’s formula telescopes into a finite sum. We also use the identity b_; = by. 


2.5 Excursions in the Weyl Chamber of Type D 


Classically, there is a simple bijection between walks with steps from {(1, 1), (1, —1)} 
which, in the first instance start and end on the axis, with no further restriction (let 
us call them bridges), and in the second instance start at the origin, and never go 
below the axis and can end at any height (Dyck prefixes). The first class restricts free 
walks by restricting the possible endpoints, and the second restricts free walks by 
restricting the size of the region. We can view this as a trade-off of restrictions. The 
simplest proof of this result passes through a third object, marked Dyck paths, which 
are objects in the intersection of both classes, but have additional markings on some 
of the down steps which touch the axis. Consider the following two maps, which 
both define bijections. A Dyck path with marked down steps is mapped to a Dyck 
prefix, by flipping all of the marked down steps into upsteps. The flipped marked 
steps become the last step at that height in the walk, and the Dyck prefix ends at a 
height equal to the number of marked steps. On the other hand, we could consider 
the entire segment of the walk ending at a marked down step, starting at the nearest 
previous up step which touched the axis. We can flip this entire segment across the 
axis. We do this for every marked step to build a bridge. The marked intermediary 
facilitates a straightforward proof, but how to mark in higher dimensions? 
The Weyl chamber of type D is the following region: 


Wolk) = (011, %2,--2,Xk) 2 Xp SB XQ Soe S Xe-1 = [I - 


An axis walk is any walk starting at the origin and ending on the x-axis. 


Theorem 4 (Courtiel, Fusy, Lepoutre and Mishna [12, Theorem 20]). For k > 1 
and n > 0, there is an explicit bijection between simple axis walks of length n stay- 
ing in Wc(k) and simple excursions of length n staying in Wp(k), starting from 
G, Se 5, 5), and ending at G, asset 5, c™). The ending x\-coordinate of a walk 
from Wc(k) corresponds to the number of steps that change the sign of x, in its 
bijective image. 

The proof of this result is a generalization of the marked Dyck path example. 
The analog to the Dyck prefix is the axis walk in the orthant space Wc (k), and the 
analog to the bridge is the excursion in the larger space Wp(k). The intermediary 
class, marked excursions in Wc(k), are defined by a less straightforward process. 
For the first bijection, Courtiel et al. map the axis walks to open arc diagrams. The 
open arcs are removed, but their position is marked. The inverse bijection is applied 
and a marked excursion results. The second bijection is slightly more complicated, 
and the reader is referred to the article for details. 
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A corollary of this result is another lattice model in bijection with standard Young 
tableaux of bounded height. 


Corollary 5 (Courtiel, Fusy, Lepoutre, Mishna [12, Corollary 21]). For n,k > 1, 
there is an explicit bijection between the standard Young tableaux of size n with 
height at most k, and the simple walks of length n staying in Wp(k), starting from 


G, gauis s, +), and ending at G. ene oy). The number of odd columns corre- 


sponds to the number of steps that chau the sign of Xx. 

It seems very likely that an explicit bijection could be found between these and the 
lazy walks. This would answer the question of Zeilberger and likely reveal a model 
for the odd cases. 

There is also potential for another generating function formula by applying the 
results of Grabiner and Magyar. 


2.6 A Collection of Bijections 


We summarize some combinatorial classes in bijection with standard Young tableaux 
of bounded height. Figure 2 illustrates some of these classes. 


Theorem 6. The set of standard Young tableaux of size n with height bounded by k 
and m odd columns is in bijection with each of the following sets: 


(1) The set of involutions of size n with m fixed points and no decreasing subsequence 
of length k + 1; 

(2) The set of oscillating tableaux of size n with height bounded by k, which start at 

the empty partition and end in a row shape X = (m); 

(if k is even) The set of open matching diagrams of length n, with m open arcs 

and with no (k/2 + 1)-crossing; 


(3 


— 


Gil, McNamara, Tirrell, and Weiner [18, Theorem 1.1] proved the equivalence 
between Class (3) in the theorem, and a set of Dyck paths where each group of ascent 
steps are decorated by a connected matching of a related size. A main feature of their 
construction is the following identity. Suppose P;, (t) denotes the generating function 
for the number of k-noncrossing perfect matchings on [27], then 


1+ P.(t?(1 — t)~*) 
1-t ; 


Yo-1(t) = 


3 Generating Function Expressions 


The generating functions of lattice walk in cones with symmetries in the stepset can 
often be expressed as a sub-series extraction of a Laurent expansion of a rational 
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function. This generalizes the result on reflectable walks of Gessel and Zeilberger 
and is at the heart of the orbit sum method of Bousquet-Mélou and Mishna [6] which 
handles a wide class of lattice walks restricted to the quarter plane. In their work, 
they start with a natural combinatorial recurrence on lattice walks, which translates 
to a functional equation satisfied by the generating function. For some models, it 
is sufficient to take a weighted sum (the namesake orbit sum) of the equations and 
isolate the target generating function with a sub-series extraction. 

In this context, the extraction operates on iterated Laurent series. For a function 
F(x,...,Xa3 t) € C(x, ..., Xa)(t) which is analytic at the origin, we denote by 
nae: : at Fo, ...,Xq} t) to be the coefficient of the term Ceo of at in 
the Laurent expansion. In this context, we view the objects to be series in ¢ variable. 
We recall the terminology that several authors use for the special case of the constant 
term with respect to a set of variables: CT,, ., F(%1,..., Xa; f) is the coefficient of 
the term x eae ae in the Laurent series expansion of F at the origin. It is a series in the 
remaining variables. It can be obtained by incrementally determining the constant 
term with respect to a single variable. Here we shall take the order of the variables as 
they are listed, for example. Such a coefficient extraction can be written as a Cauchy 
integral. 

A related operator is the diagonal. The (central) diagonal AF (xo, x1,..., Xq) of 
a formal power series is the univariate sub-series defined 


AF (%,...,%¢) = A oz Pewniget as = LF. 


Lipshitz [27] proved that the diagonal of a D-finite function‘ is also D-finite. Since 
rational functions are D-finite, diagonals of rational functions are also D-finite. The 
expressions we consider here are all diagonals of rational functions and hence are 
all D-finite by construction. This provides an alternate proof of the D-finiteness of 
Y;(t). 

Bousquet-Mélou [7] used an orbit sum to directly derive generating function 
expressions for standard Young tableaux of bounded height. Her generating func- 
tions also mark the number of odd columns, and other parameters are also readily 
accessible. In that same paper, she determines a nice functional equation proof of 
the hook length formula. The starting point is a simple recursive construction for 
standard Young tableaux: a tableau of size n + 1 is obtained from a tableau of size 
n by adding a cell to the jth row, unless the jth row is already the same length as 
the 7 — 1 row. This translates into a very straightforward functional equation for the 
generating function 


Flu)= d Pua ae, 


tek 


4A (multivariate) function is D-finite if the set of all its partial derivatives spans a vector space of 
finite dimension. 
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The equation is defined in term of F;(u), the generating function for those standard 
Young tableaux such that A;_; = Aj: 


k 


F(u) =1+u;F(u) + So uj(FW) — F;(u)). 
j=2 


She applies a kernel method argument to this functional equation to recover 
MacMahon’s formula for f>. 

She has generating function expressions in her Propositions 9 and 10. It is her 
Proposition 11 that accounts for the additional parameter of interest. We note that 
she works with the ordinary generating function, 


Y, (th= > > tia todd columns in \ t”. 
n AEn 
eQ)<k 


Theorem 7 (Bousquet-Mélou [7]). For k = 2€, the ordinary generating function of 
standard Young tableaux with height bounded by k where t marks the length and x, 
marks the number of odd columns is 


— det(x;* — x/)1<ij<e 1 


¥,(x1; t) = CTy,,...x, (10) 


L—tQ tap) +--+ xe + xy") xox? of? 


The orbit sum strategy can be used to determine an expression for the ordinary 
generating function for simple walks in Wc(k) ending on the axis. This gives the 
following result. 


Theorem 8 (Burrill, Melczer and Mishna [9]). The ordinary generating function, 
Y,(t) for Young tableaux of height bounded by k satisfies the formula formula 
PANY 54 (t) = 
; zoe eset ee + D Myc jcice Gi — 2s) Gizj — D- Three? - D 
Lops eer tap) +e ety) , 


These formulas are not necessarily easier to compute or interpret than the others, 
but they do have a few implications. They do provide a proof that the generating 
function is D-finite. Furthermore, we see a second proof that y;,(m) can be expressed 
as binomial sums, and potentially a different route to obtain such expressions, using 
the work of Bostan, Lairez and Salvy [5]. They may be suitable for analysis by the 
methods of Pemantle and Wilson [41], which determine asymptotic formulas for 
coefficients of functions which are expressed as diagonals of rational functions. 

Since it is the generating function for Motzkin numbers, Y3(t) is algebraic. 
Because the asymptotics of the coefficients are incompatible with algebraicity, Y4(t) 
is not. For which k is the ordinary generating function ¥; (t) algebraic? 
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3.1 Differential Equations 


Once we know that a generating function is D-finite, it is natural to ask about dif- 
ferential equations that it satisfies. Bergeron, Favreau and Krob [3] generated many 
conjectures about the order of the differential equation satisfied by Y;(t) from com- 
puter experiments, and some analysis of its expression as a determinant of a matrix 
of modified Bessel functions. 

Proposition 1 in [2] states the dimension of the vector space over the field C(t) 
of rational functions in ¢ spanned by Y;(t) and all its derivative is bounded by LEI. 


Conjecture 9 (Bergeron, Favreau, Krob [3]). For each k, there are polynomials 
Pm(t) of degree at most LE such that Y;(t) is a solution of a linear differential 
equation order at most LE + 1 with coefficients Py (t). 

They have verified this conjecture for k < 11. 

We can use recent advances in symbolic computation and the diagonal expression 
in Theorem 8 to bound the order and degree of the differential equations satisfied 


by the ordinary generating function Y; (t). In particular, the following theorem of 
Bostan, Lairez, and Salvy [4] can be explicitly applied. 


Theorem 10 (Bostan, Lairez, and Salvy [4]). Let R(z,,..., Za, t) = A(z, t)/B, t) 
€ Q(t)(z1,.--, Z¢), be a rational function with multidegree bounds 


n, := max(deg, B, deg, A+ d+ 1) n; = max(deg, A, deg, B). 


Then there exists an annihilating differential equation L for the integral 
P(t):= § R(z, t)dz, 
" 


where y is any n-cycle in C" on which R is continuous when t ranges over some 
connected open set U C C (note that L is independent of y). Furthermore the order 
of £ is at most né and the degree of L is at most (3n34 + nt) (ny. 


We write a diagonal as an integral: 


1 d F pete iy 
ie (<1, ca/e1, 23/22, + 2alzat tea) yyy 
*y 


AF(z, t) = ( - 
Wi te 
by the multivariate Cauchy residue theorem, for an appropriate n-cycle y around the 
origin. 
In the case of Yj (t), we compute an upper bound of 2d? — 3d + 1 on the total 
degree of the denominator in the z variables and a bound of 2d? — 4d — 2 on the 
degree of the numerator and implies the following result, computed by Melczer. 
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Proposition 11. The generating function for the number of standard Young tableaux 
of height at most k satisfies a linear differential equation of order at most (2k* — 
3k + 1)* and degree at most 


5 
(Fcc — 3d + 1)*4 + (2d? — 3d + »') e, 


3.2 Asymptotics 


Regev [29] determined asymptotic expansions for the number of Young tableaux of 
bounded height. He explicitly deduced asymptotics for the k = 3 case as a particular 
extraction: 


(n+ Iys(n) = x N@ t+ 1/x+ "1 ~ 2 =e 


By aslightly more general argument, he showed 


k-1 


Yor) ~nsoo (2/m)*/?(2k)" (k/nye TT iy!. (12) 
i=0 


The asymptotics of lattice walks in cones have been well studied. The tour de 
force of Denisov and Wachtel [13] describes a collection of very comprehensive 
results. The formulas given in [13, Theorem 6] should be applied here, for example 
to the lazy walks and to the excursions in Wp(k). 

Grabiner [24] used his formulas for walks in Weyl chambers to find the asymp- 
totics of the probability that a randomly chosen standard Young tableau of size n 
with at most f rows contains a given subtableau. This is equivalent to counting walks 
that have visited a particular point—there might be similar results to be extracted by 
considering other lattice models. 

The asymptotic formula that has been described so far are valid for fixed k, as n 
tends to infinity. It is open to develop formulas when k is a function of n. 


4 Restricting Increasing Subsequences in Permutations 


Motivated by algebraic interpretations, Regev [29] considered the quantity 


wm= Vu 


AEPE 


What can be said of the generating functions 
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B B 
Dex at" Yo yeartu?? 


When (@ = 2, this counts permutations with restricted longest increasing sub- 
sequence, and it is very well studied, with many relevant connections to lattice 
walks [39, 40]. Define u;(m) to be the number of pairs of Young tableaux of the 
same shape with at most k rows. By the Robinson—Schensted correspondence, this 
is the number of permutations in G,, with no (k + 1)-increasing subsequence. For 
every k > 1, Gessel [16] proved the formula 


ux (Nn) 
a niepn det Cs eee , (13) 


n>0 


where the b; are the Bessel function evaluations defined earlier. 

A combinatorial proof of this expression has been given by Gessel et al. [15] via 
simple walks ending at so-called Toeplitz points, and Xin [42] did a combinatorial 
derivation based on arc diagrams, and yet another constant term extraction. The 
permutations satisfy a nice combinatorial recursion, and, in a manner similar to her 
solution for involutions, Bousquet-Mélou [7] determines functional equations that 
can be resolved using a kernel method approach. 

She determined [7, Proposition 13] the following expression for the ordinary 
generating function of permutations avoiding the pattern 12...m (m+ 1): 


m i 


Pe ay: tad bee 
Ue(t) = Yo ue)" = CT ogre EY ishcism TA, 
n=0 1-t (x —) i=0 j=1 1- x; 
(14) 
Bergeron and Gascon found the differential equations satisfied by the exponential 
generating functions fork < 11. Itremains open to answer if ( y (n)) is the counting 
sequence for any easily characterizable combinatorial family, perhaps as a restricted 
or decorated family of permutations. The argument of Gessel [16] applies to prove 
that the sequence is P-recursive for positive integer 3, and probably also a diagonal 
of a rational function. Can we usefully bound the annihilating differential operators 
or determine asymptotic formulas for arbitrary 3? 
Wilf [40] deduced U2; (t) = Yo, (t) Yo; (—t). From this, it follows 


2 2 
( ua = dX ( ") (—1)" yor (1) yx (2n — 1). 


Is there a lattice path interpretation of this identity? Are there identities for other 3 
values? 
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5 Other Directions 


5.1 Using Kronecker Coefficients 


The Kronecker product of symmetric functions gives an important connection to rep- 
resentation theory. In particular, the Kronecker product of two symmetric functions 
in the Schur function basis determines the multiplicities of irreducible characters in 
this tensor product. 

The following Schur function identity for the Kronecker product of two Schur 
functions (denoted by *) was shown by Brown, van Willigenburg and Zabrocki [8]: 


S(n,n—1) * S(n.n-1) = yE Sy. 


AF2n-1 
£0) <4 


Tewari [37] manipulates this formula to deduce a closed form for the number of 
Young tableaux with height exactly 5, under the additional constraint of A5 = 1. His 
Theorem 7.4 is a simple sum of Motzkin numbers (recall M,, = y3(n)) and pairs of 
Catalan numbers (recall also the formula of ya): 


n(n + 2) 
> fl = (ee) Cr Cn41 — Cy + Mon, 


(n+ 1) 
~~ f° — ( 5) ) C? = CaCn+1 + Mon-1. 


It could be straightforward to find a combinatorial interpretation of his formula, 
which then could be generalized to higher dimensions. It also suggests that perhaps 
some of the larger summations could be simplified. This could be key to finding a 
useful expressions for yo(7). 


5.2 Other Classes in Bijection 


As Gouyou-Beauchamps noted in [23], the numbers that appear in y4(1) also appear 
in the enumeration of planar maps and alternating Baxter permutations. Baxter per- 
mutations are a class of pattern avoiding permutations that are very combinatorially 
rich. They also have a bijection to lattice path models, using the machinery of arc 
diagrams as an intermediary class. It might be possible to directly connect these 
classes. 
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5.3. Shadow Diagrams 


Some of the lattice walk bijections use tableau insertion and deletion in more than 
one stage. Perhaps there exist more economical, or direct bijections. The shadow 
diagrams of Viennot may play an important role in such a simplification. 


5.4 Random Tableaux 


There have been several recent works on generating random walks. One application 
is to convert this to a generator for random tableaux. Which of the above bijections 
has smallest complexity? 

Grabiner [24] was able to use lattice walk results to determine distributions of 
subtableaux in Young tableaux. It seems that there should be more results along this 
vein with each of the lattice walk representations. 


5.5 Semi-standard Young Tableaux 


Okada [28] proved some results connecting counting sequences of generalized oscil- 
lating tableaux and semi-standard tableaux using techniques from representation 
theory. In particular, his Theorem 5.3 is a list of results on equinumerous classes of 
tableaux of bounded height that are well suited for more bijective explanations. 

Krattenthaler’s results [26] on semi-standard tableaux replace single steps in oscil- 
lating tableaux with jumps by horizontal strips. Perhaps they can connect here using 
lattice walks with longer steps or diagonal steps. In any case, these two works should 
be connected more explicitly. 
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A Tale of Centrally Symmetric Polytopes ®) 
and Spheres ce 


Isabella Novik 


Abstract This paper is a survey of recent advances as well as open problems in the 
study of face numbers of centrally symmetric simplicial polytopes and spheres. The 
topics discussed range from neighborliness of centrally symmetric polytopes and the 
Upper Bound Theorem for centrally symmetric simplicial spheres to the Generalized 
Lower Bound Theorem for centrally symmetric simplicial polytopes and the lower 
bound conjecture for centrally symmetric simplicial spheres and manifolds. 


1 Introduction 


The goal of this paper is to survey recent results related to face numbers of cen- 
trally symmetric simplicial polytopes and spheres. To put things into perspective, we 
start by discussing simplicial polytopes and spheres without a symmetry assumption. 
The classical theorems of Barnette [10] and McMullen [48], known as the Lower 
Bound Theorem (LBT, for short) and the Upper Bound Theorem (or UBT), assert 
that among all d-dimensional simplicial polytopes with n vertices a stacked polytope 
simultaneously minimizes all the face numbers while the cyclic polytope simulta- 
neously maximizes all the face numbers. Furthermore, the same results continue to 
hold in the generality of (d — 1)-dimensional simplicial spheres (see [9, 73]). It is 
also worth mentioning that in the class of simplicial spheres of dimension d — | > 3 
with n vertices, the (boundary complexes of the) stacked polytopes are the only min- 
imizers [37]. On the other hand, the maximizers are precisely the |d/2|-neighborly 


Research is partially supported by NSF grants DMS-1361423 and DMS-1664865, and by Robert 
R. and Elaine F. Phelps Professorship in Mathematics. This material is based on work supported by 
the National Science Foundation under Grant No. DMS-1440140 while the author was in residence 
at the Mathematical Sciences Research Institute in Berkeley CA, during the Fall 2017 semester. 


I. Novik 

Department of Mathematics, University of Washington, 
Seattle, WA 98195-4350, USA 

e-mail: novik @math.washington.edu 


© The Author(s) and the Association for Women in Mathematics 2019 305 
H. Barcelo et al. (eds.), Recent Trends in Algebraic Combinatorics, 

Association for Women in Mathematics Series 16, 

https://doi.org/10.1007/978-3-030-05141-9_9 


306 I. Novik 


spheres—a set that, in addition to the (boundary complex of the) cyclic polytope, 
includes many other simplicial spheres, see, for instance, a recent paper [63]. 

In fact, much more is known: the celebrated g-theorem of Billera—Lee [15] and 
Stanley [74] provides a complete characterization of all possible f-vectors of sim- 
plicial polytopes. The equally celebrated g-conjecture posits that the same charac- 
terization is valid for the f-vectors of simplicial spheres. (The f-vector encodes 
the number of faces of each dimension of a simplicial complex.) In other words, at 
least conjecturally, the f-vector cannot differentiate between simplicial polytopes 
and simplicial spheres. 

Let us now restrict our world and consider all centrally symmetric (cs, for short) 
simplicial polytopes and all centrally symmetric simplicial spheres. What is known 
about the f-vectors of these objects? Are there cs analogs of the LBT and UBT, 
and ultimately of the g-theorem and g-conjecture, respectively? Surprisingly very 
little is known, and not for the lack of effort. There is an analog of the LBT for 
cs simplicial polytopes established by Stanley [75] as well as a characterization of 
minimizers [41], and while it is plausible that the same results continue to hold for 
all cs simplicial spheres, the proofs remain elusive. 

Still, the most mysterious and fascinating side of the story comes from trying to 
understand the upper bounds: in contrast with the situation for simplicial polytopes 
and spheres without a symmetry assumption, the existing upper bound-type results 
and conjectures indicate striking differences between f-vectors of cs polytopes and 
those of cs spheres. In other words, cs spheres and polytopes do not look alike f-wise, 
even though their non-cs counterparts do! For instance, the (appropriately defined) 
neighborliness of cs polytopes is quite restrictive [46], and a cs d-dimensional poly- 
tope with more than 2 vertices cannot be even 2-neighborly; yet, by a result of 
Jockusch [36], there exist cs 2-neighborly simplicial spheres of dimension 3 with 
any even number n > 8 of vertices. Thus, for a sufficiently large n, the maximum 
number of edges that a cs 4-dimensional polytope with n vertices can have differs 
from the maximum number of edges that a cs simplicial sphere of dimension 3 with 
n vertices can have; moreover, the former quantity (or even its asymptotic behavior) 
is unknown at present. This indicates how very far we are from even posing a plau- 
sible (sharp) upper bound conjecture for cs d-dimensional polytopes with d > 4. At 
the same time, Adin [2] and Stanley (unpublished) provided upper bounds on face 
numbers of cs simplicial spheres of dimension d — 1 with n vertices; these bounds 
are attained by a cs |d/2]|-neighborly sphere of dimension d — | with n vertices 
assuming such a sphere exists. 

Aside from being of intrinsic interest, additional motivation to better understand 
the f-vectors of cs polytopes arises from the recently discovered tantalizing connec- 
tions (initiated by Donoho and his collaborators, see, for instance, [22, 23]) between 
cs polytopes with many faces and seemingly unrelated areas of error-correcting codes 
and sparse signal reconstruction. Furthermore, any cs convex body in R? is the unit 
ball of a certain norm on R¢. As a result, methods used in the study of face numbers 
of cs complexes, at present, involve not only commutative algebra (via the study of 
associated Stanley—Reisner rings) but also a wealth of techniques from geometric 
analysis. Yet, many problems remain notoriously difficult. 
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The goal of this paper is thus to survey a few of the known results on f-vectors 
of cs simplicial polytopes and spheres, showcase several existing techniques, and, 
most importantly, present many open problems. It is our hope that collecting such 
problems in one place will catalyze progress in this fascinating field. For a quick 
preview of what is known and what is not, see the following table. 


non-cs cs 
simplicial |simplicial |same? simplicial |simplicial |same? 
polytopes |spheres polytope {spheres 
UBT |./ a/ yes no plau-/known [no 
sible con-|bounds 
jecture conjec- 
turally 
sharp 
LBT |./ ay yes ni conjecture |conjecturally yes 
GLBT]./ conjecture |conjecturally yes|./ conjecture |conjecturally yes 


The rest of the paper is structured as follows. In Sect.2, we set up notation and 
recall basic definitions pertaining to simplicial polytopes and spheres. Section3 is 
devoted to neighborliness of cs polytopes. This leads to discussion of upper bound- 
type results and problems on face numbers of cs polytopes, see Sect.4. Section 5 
deals with neighborliness and upper bound-type results for cs simplicial spheres. 
Section 6 takes us into the algebraic side of the story: there we present a quick review 
of Stanley—Reisner rings—the major algebraic tool in the study of face numbers, 
sketch the proof of the classical UBT along with Adin—Stanley’s variant of this result 
for cs spheres, and prepare the ground for the following sections. Sections7 and 8 
are concerned with the lower bound-type results and conjectures. Specifically, in 
Sect. 7 we discuss acs analog of the Generalized LBT for cs simplicial polytopes—a 
part of the story that is most well understood, while in Sect. 8 we consider a natural 
conjectural cs analog of the LBT for spheres, manifolds, and pseudomanifolds. We 
close with a few concluding remarks in Sect. 9. 


2 Preliminaries 


We start with outlining basic definitions and notation we will use throughout the 
paper. A polytope is the convex hull of a set of finitely many points in R¢. One 
example is the (geometric) d-simplex defined as the convex hull of an arbitrary set 
of d + 1 affinely independent points in R¢. A (proper) face of any convex body K 
(e.g., a polytope) is the intersection of K with a supporting affine hyperplane; see, 
for example, Chapter II of [11]. A polytope P is called simplicial if all of its (proper) 
faces are simplices. The dimension of a polytope P is the dimension of the affine 
hull of P. We refer to d-dimensional polytopes as d-polytopes and to i-dimensional 
faces as i-faces. 
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Apolytope P C Ris centrally symmetric (cs, for short) if P = —P;thatis,x € P 
if and only if —x € P. An important example of a cs polytope is the d-dimensional 


cross-polytope C7 = conv(+p}, =p2,..., pa), where p;, po, ..., Pa are points in 
IR¢ whose position vectors form a basis for R@. If these position vectors form the 
standard basis of R“, denoted {e;,..., ea}, then the resulting polytope is the unit 


ball of the £'-norm; we refer to this particular instance of the cross-polytope as the 
regular cross-polytope. 

A simplicial complex A on a (finite) vertex set V = V(A) is a collection of 
subsets of V that is closed under inclusion; an example is the (abstract) simplex 
on V, V:= {F : F CV}. The elements of A are called faces, and the maximal 
under inclusion faces are called facets. The dimension of a face F € A is dim F = 
|F'| — 1, and the dimension of A is dim A := max{dim F : F € A}. Thek-skeleton 
of A, Skel, (A), is a subcomplex of A consisting of all faces of dimension < k. The 
f-vector of a simplicial complex A is f(A) := (f-1(A), fo(A),.--, faima(A)), 
where f; = f;(A) denotes the number of i-faces of A; the numbers /; are called the 
f-numbers of A. 

Each simplicial complex A admits a geometric realization ||A|| that contains a 
geometric i-simplex for each i-face of A. A simplicial complex A is a simplicial 
sphere (simplicial ball, or simplicial manifold, respectively) if || A|| is homeomorphic 
to a sphere (ball, or closed manifold, respectively). If P is a simplicial d-polytope, 
then the empty set along with the collection of the vertex sets of all the (proper) faces 
of P is a simplicial sphere of dimension d — | called the boundary complex of P; 
it is denoted by OP. While it follows from Steinitz’ theorem that every simplicial 
2-sphere can be realized as the boundary complex of a simplicial 3-polytope, for 
d > 4, there are many more combinatorial types of simplicial (d — 1)-spheres than 
those of boundary complexes of simplicial d-polytopes, see [38, 57, 65]. 

A simplicial complex A is centrally symmetric (or cs) if it is equipped with a 
simplicial involution ¢ : A — A such that for every non-empty face F « A, d(F) 4 
F.. We refer to F and ¢(F) as antipodal faces. For instance, the boundary complex of 
any cs simplicial polytope P is a cs simplicial sphere with the involution ¢ induced 
by the map ¢(v) = —v on the vertices of P. 

A simplicial complex A is k-neighborly if every set of k of its vertices forms a 
face of A. Equivalently, a simplicial complex A with n vertices is k-neighborly if its 
(k — 1)-skeleton coincides with the (k — 1)-skeleton of the (1 — 1)-simplex. Since 
in a cs complex, a vertex and its antipode can never form an edge, this definition 
of neighborliness requires a suitable adjustment for cs complexes. We say that a 
cs simplicial complex A is k-neighborly if every set of k of its vertices, no two of 
which are antipodes, forms a face of A. Equivalently, acs simplicial complex A on 2m 
vertices is k-neighborly if its (k — 1)-skeleton coincides with the (k — 1)-skeleton of 
OC; . In particular, if A is a k-neighborly simplicial complex, then f;_;(A) = (4 i) 
for all 7 < k, while if A is a cs k-neighborly simplicial complex, then fj_;(A) = 
21 1”) for all j < k. 

Itis worth mentioning that similar definitions apply to general (i.e., not necessarily 
simplicial) polytopes. Specifically, a polytope P is k-neighborly if every set of k of 
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its vertices forms the vertex set of a face of P; acs polytope P is k-neighborly if 
every set of k of its vertices, no two of which are antipodes, forms the vertex set of 
a face of P. In the next two sections, we work with general cs polytopes. 


3 How Neighborly Can a cs Polytope Be? 


Our story begins with the cyclic polytope, Ca(n), which is defined as the convex hull 
of n > d +1 distinct points on the moment curve {(t, t?7,...,f%) ER: te R} or 
on the trigonometric moment curve {(cost, sint, cos 2t, sin2t,..., coskt, sinkt) € 
R** : ¢ € R}, assuming d = 2k. Both types of cyclic polytopes were investigated 
by Carathéodory [19] and later by Gale [27], who, in particular, showed that the two 
types are combinatorially equivalent (assuming d is even) and independent of the 
choice of points. In fact, the two types are projectively equivalent, see [84, Exercise 
2.21]. These polytopes were also rediscovered by Motzkin [32, 53] and many others; 
see books [11, 84] for more information on these amazing polytopes. The properties 
of the cyclic polytope Cy(n) that will be important for us here are: it is a (non-cs) 
simplicial d-polytope with n vertices; furthermore, it is |d/2|-neighborly for all 
n>d+l. 

The existence of cyclic polytopes motivated several questions, among them: do 
there exist cs d-polytopes (apart from the cross-polytope) that are |d/2|-neighborly 
or at least “highly” neighborly? This section discusses the state-of-affairs triggered 
by this question. 

It became apparent from the very beginning that the answer is likely to be both 
interesting and complicated: several works from the sixties indicated that in contrast 
to the general case, the neighborliness of cs polytopes might be rather restricted. 
Specifically, Griinbaum showed in 1967 [31, p. 116] that while there exists a cs 2- 
neighborly 4-polytope with 10 vertices, no cs 4-polytope with 12 or more vertices 
can be 2-neighborly. This observation was extended by McMullen and Shephard [51] 
who proved that while there exists a cs |d/2|-neighborly d-polytope with 2(d + 1) 
vertices, a cs d-polytope with at least 2(d + 2) vertices cannot be more than [(d + 
1)/3]-neighborly. A cs |d/2|-neighborly d-polytope with 2(d + 1) vertices is easy 
to construct: for instance, conv ( +te,,te,. eds a 1 ei) does the job. On 
the other hand, to show that a cs d-polytope with Xd + 2) vertices can only be |(d + 
1)/3]|-neighborly and to construct cs polytopes achieving this bound, McMullen and 
Shephard [51] introduced and studied cs transformations of cs polytopes—a cs analog 
of celebrated Gale transforms. (See [31, Section 5.4] and [84, Chapter 6] for a detailed 
description of Gale transforms and their applications.) 

Let k(d,n) denote the largest integer k such that there exists a cs d-polytope 
with 2(n + d) vertices that is k-neighborly. Influenced by their result, McMullen 
and Shephard [51] conjectured that, in fact, k(d,n) < \(d+n-—1)/(n+1)] for 
all n > 3. Their conjecture was subsequently refuted by Halsey [33] and then by 
Schneider [69], but only for d >> n. Namely, Schneider showed that 
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k(d,3 k(d, 
lim inf Whe) 1—2-' and liminf kim) > 0.2390 for all n. 
d+3 d+n 


doo doo 


On the other hand, Burton [18] proved that a cs d-polytope with a sufficiently large 
number of vertices ((d /2)“/*) indeed cannot be even 2-neighborly. Burton’s proof is 
surprisingly short and simple: it relies on John’s ellipsoid theorem, see for instance [8, 
Chapter 3], along with a quantitative version of the observation that any sufficiently 
large finite subset of the Euclidean unit sphere contains two points that are very close 
to each other. 

To the best of the author’s knowledge, McMullen—Shephard’s values k(d, 1) = 
L$] andk(d, 2) = [| (see [51]) remain the only known exact values of k(d, n) for 
alld. In particular, k(4, 1) = 2andk(4, 2) = 1, while k(5, 1) = k(5, 2) = 2. Further- 
more, the main result of a very recent paper [60] implies that k(5, 3) = k(5, 4) = 2, 
but it appears unknown whether k(5, 5) equals 2 or 1. On the other hand, the asymp- 
totics of k(d, n) is now well understood: 


Theorem 3.1. There exist absolute constants C,, C2 > 0 independent of d and n 
such that Cd Cod 

— og <ed,n) <14§ —* =. 

1+ log “+4 1 + log “*¢ 


d 

Theorem 3.1 is due to Linial and Novik [46]. A dual version of the lower bound part 
of this theorem was also independently established by Rudelson and Vershynin [67]. 

Two extreme, and hence particularly interesting, cases of Theorem 3.1 deserve a 
special mention: the case of k(d,n) proportional to d and the case of k(d,n) = 1. 
Donoho [22] proved that there exists p > 0 such that for large d, the orthogonal pro- 
jection of the 2d-dimensional regular cross-polytope onto a d-dimensional subspace 
of R*“, chosen uniformly at random, is at least | d|-neighborly with high proba- 
bility and provided numerical evidence that p > 0.089. In other words, for large d, 
k(d, d) = pd. (The estimates from [46] guarantee that p > 1/400.) As for the other 
extreme, Theorem 3.1 shows that the largest number of vertices in a cs 2-neighborly 
d-polytope is e°™, In fact, the following more precise result is known, see [46, 
Theorem 1.2] and [12, Theorem 3.2]. 


Theorem 3.2. A cs 2-neighborly d-polytope has at most 24 vertices, i.e., k(d, 24—! + 
1 — d) = 1. On the other hand, for any even d > 6, there exists a cs 2-neighborly 


d-polytope with 23" /3 — 1) vertices. 


While this chapter awaited its publication, the author proved in [60] that for every 
d > 2, there exists a cs 2-neighborly d-polytope with 27~! + 2 vertices. 

Our discussion of k(d, n) is summarized in a table below. 

We devote the rest of this section to pointing out some of the main ideas used 
in the proofs. In particular, the proof of the fact that a cs 2-neighborly d-polytope 
has at most 2¢ vertices is so short that we cannot avoid the temptation to provide 
it here. For a polytope P C R¢@ and a vector a € R“, define P, := P + a to be the 
translation of P by a, where “+” denotes the Minkowski addition. The first step in 
the proof is the following simple observation from [14]: 
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n k(d,n) 


5: 
2 [2 


3 
d proportional to d 


<277 41 —dl>2 
S271 +1-d[1 


Lemma 3.3. Let P C R¢ be acs d-polytope, and let u and v be vertices of P, so 
that —v is also a vertex of P. If the polytopes P, and P, have intersecting interiors 
then the vertices u and —v are not connected by an edge. Consequently, if P is a 
cs 2-neighborly polytope with vertex set V, then the polytopes {P, : v € V} have 
pairwise disjoint interiors. 


Proof: The assumption that int (P,,) N int (P,) 4 % implies that there exist x, y € 
int (P) such that x + u = y + v, or equivalently, that (y — x)/2 = (u — v)/2. Since 
P is centrally symmetric, and x, y € int (P), the point g := (y — x)/2 is an interior 
point of P. As gq is also the barycenter of the line segment connecting u and —v, this 
line segment is not an edge of P. 


The rest of the proof that a cs 2-neighborly d-polytope has at most 2¢ vertices 
utilizes a volume trick that goes back to Danzer and Griinbaum [21]. If P C R?¢ is 
acs 2-neighborly d-polytope with vertex set V, then by Lemma 3.3, the polytopes 
{P, : v € V} have pairwise disjoint interiors. Therefore, 


Vol (\_J P.) = }¢ Vol (P,) = |V| - Vol (P). 


veV veV 


On the other hand, since for v € V, P, = P+ C 2P, it follows that U Mcp Pye 
2P, and so 
Vol ( U P,) < Vol(2P) = 24 - Vol(P). 


veV 


Comparing these two equations yields |V| < 24, as desired. 

The proof of the upper bound part of Theorem3.! relies on a more intricate 
application of the same volume trick. Let P be acs k-neighborly d-polytope, where 
k = 2s for some integer s. We say that a family F of (s — 1)-faces of P is good if 
every two elements F # G of F satisfy the following conditions: F' and G share at 
most s/2 vertices, while F and —G have no common vertices. To obtain an upper 
bound on the size of the vertex set V of P in terms of d and k, one first observes that 
if F is a good family, F # G are in F, and by and bg are the barycenters of F and 
G, then the polytopes P + 2b, and P + 2bg have disjoint interiors (cf. Lemma 3.3). 
One then uses a simple counting argument to show that there is a relatively large (in 
terms of d,s, and |V|) good family F. Since the translates {P+ 2br : F € F}ofP 
have pairwise disjoint interiors and are all contained in 3 P, the volume trick yields 
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a desired upper bound on |F |, and hence also on |V|; see the proof of Theorem 1.1 
in [46] for more details. 

The proof of the lower bound in Theorem 3.1 is based on studying the cs trans- 
forms of cs polytopes introduced in [51] and on “high-dimensional paradoxes” such 
as KaSin’s theorem [40] and its generalization due to Garnaev and Gluskin [28]. 
Specifically, KaSin’s theorem asserts that there is an absolute constant C (for instance, 
32 does the job, see [8, Lecture 4]) such that for every d, IR has a d-dimensional 
subspace, Ly, with the following property: the ratio of the £?-norm to the £'-norm of 
any nonzero vector x € Lg is in the interval le salt we refer to such a subspace 


as a KaSin subspace. Via cs transforms, d-dimensional subspaces of R~“ correspond 
to cs d-polytopes with 4d vertices; furthermore, a careful analysis of cs transforms 
shows that the polytopes corresponding to Kain subspaces are k-neighborly with k 
proportional to d. 

More generally, a theorem due to Garnaev and Gluskin [28] quantifies the extent 
to which an n-dimensional subspace of R”*“ can be “almost Euclidean” (meaning 
that the ratio of the £?-norm to the ¢!-norm of nonzero vectors remains within certain 
bounds, more precisely, it is < C pelaster ta for some absolute constant C ). Via 


cs transforms, n-dimensional subspaces of R”+“ correspond to cs d-polytopes with 
2(n + d) vertices, and the “almost Euclidean” subspaces correspond to cs polytopes 
with neighborliness given by the lower bound in Theorem 3.1, see [46] for technical 
details. 

The proof of the Garnaev—Gluskin result and hence also of the lower bound part 
of Theorem 3.1 is probabilistic in nature: it does not give an explicit construction of 
neighborly cs polytopes, but rather shows that they form a set of positive probability 
in a certain probability space. Indeed, it is an interesting open question to find an 
explicit construction for highly neighborly cs polytopes that meet the lower bound 
in Theorem 3.1. We discuss some known explicit constructions (for instance that of 


acs 2-neighborly d polytope with ~ a vertices) in the next section. It would also 
be extremely interesting to shed some light on the exact values of k(d, n): 


Problem 3.4. Determine the exact values of k(d,n), or at least find the value of 
no(d) := min{n : k(d,n) = 1}, that is, find the number n starting from which a cs 
d-polytope with 2(d + n) vertices cannot be even 2-neighborly. 


4 Toward an Upper Bound Theorem for cs Polytopes 


The fame of the cyclic polytope comes from the Upper Bound Theorem (UBT, for 
short) conjectured by Motzkin [53] and proved by McMullen [48]. It asserts that 
among all d-polytopes with n vertices, the cyclic polytope Cy(n) simultaneously 
maximizes all the face numbers. The goal of this section is to summarize several 
upper bound-type results and problems for cs polytopes motivated by the UBT. 
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What is the maximum number of k-faces that a centrally symmetric d-polytope 
with WN vertices can have? While our discussion in the previous section suggests that 
at present we are very far from even posing a plausible conjecture, certain asymp- 
totic results on the maximum possible number of edges are known. Specifically, the 
following generalization of Theorem 3.2 holds. This result is in sharp contrast with 
the fact that f{(Cg(n)) = (3) as long asd > 4. 


Theorem 4.1. Let d > 4. If P C R¢ is acs d-polytope on N vertices, then 
N2 
fi(P) SU - 2). 


On the other hand, there exist cs d-polytopes with N vertices (for an arbitrarily large 
N) and at least ‘al _ 37 l4/2—1)) () x(1- sa) . we edges. 


The first part of Theorem4.1 was established by Barvinok and Novik in [14, 
Proposition 2.1]; its proof relies on an extension of the argument discussed in the 
previous section to obtain an upper bound on the number of vertices that a cs 2- 
neighborly d-polytope can have and more specifically on Lemma 3.3. The idea, very 
roughly, is as follows: each of the N vertex translates of P, P, for u € V, has the 
same volume as P, and all of them are contained in the polytope 2 P,, whose volume 
is 27 Vol (P). Hence, on average, an interior point of 2P belongs to N/24 (out of N) 
sets int (P,,). Therefore, on average, int (P,,) intersects with the interiors of at least 
N/2¢ — 1 other vertex translates of P. Consequently, by Lemma 3.3, the average 
degree of a vertex of P in the graph of P is < N(1 — 2~“). This yields the desired 
upper bound on the number of edges of P. 

The second part of Theorem4.1 is due to Barvinok, Lee, and Novik [12]. The 
proof is based on an explicit construction whose origins can be traced to work of 
Smilansky [70]. To discuss this part, we start by recalling that the cyclic polytope 
C,(n) is defined as the convex hull of 7 points on the moment curve or, if d = 2k is 
even, on the trigonometric moment curve J; : R > R*, where 


T(t) = (cost, sint, cos 2t, sin2t,..., coskt, sinkt). 


In an attempt to come up with a cs analog of the cyclic polytope, Smilansky [70] 
(in the case of k = 2), and Barvinok and Novik [14] (in the case of arbitrary k) 
considered the symmetric moment curve, U, : R + R**, defined by 


U;(t) = (cost, sint, cos 3t, sin 3t,..., cos(2k — 1)t, sin(2k — 1)f). 


Since U;(t) = U;(t + 27) for allt € R, from this point on, we think of U;, as defined 
on the unit circle S = R/27Z. The name symmetric moment curve is explained 
by an observation that for all t ¢ S, ¢ and ¢+ 7 form a pair of opposite points 
and Ux(t + 7) = —U;,(t). A bicyclic polytope is then defined as the convex hull of 
{U,(t) : t € X}, where X C Sis a finite subset of S; we will also assume that X is a 
cs subset of S. 
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The papers [70] (in the case of k = 2), and [14] along with [79] (in the case of 
arbitrary k) study the edges of bicyclic polytopes. In particular, when k = 2, the 
following result established in [70] (see also [14]) holds. Recall that a face of a 
convex body K is the intersection of K with a supporting hyperplane. 


Theorem 4.2. Let [ CS be an open arc of length 27/3 and let ty, ty € T. Then 
the line segment conv(U2(t;), U2(t2)) is an edge of the 4-dimensional convex body 
By := conv(U2(t): t € S). 


One immediate consequence is 


Corollary 4.3. Let X = {0, 7/2, 7, 37/2} CS, lets > 2 bean integer, and let X, be 
acs subset of S obtained from X by replacing each € X witha cluster of s points all 
of which lie on a small arc containing T. Then the polytope Qs := conv(U2(t) : t € 
X;) is acs 4-polytope that has N := 4s vertices and at least | : N(GN -—1l)e 3(7) 


4 
edges. 


Indeed, it follows from Theorem 4.2 that each vertex of Q, is connected by an edge 
to all other vertices of Q, except possibly those coming from the opposite cluster, 
yielding the result. 

Denote by fmax(d, N; k — 1) the maximum possible number of (k — 1)-faces 
that a cs d-polytope with N vertices can have. From the above discussion, we infer 
that 3 . 

: : wee O(N) < fmax (4, N; 1) < 2 : = 

4 2 16 2 
These are currently the best-known bounds on the maximum possible number of 
edges that a cs 4-polytope with N vertices can have. 

Perhaps somewhat surprisingly, fork > 2, bicyclic polytopes do not have a record 
number of edges. However, the symmetric moment curve is used in the following 
construction that produces cs polytopes with the largest known to-date number of 
edges. 

Let m > 1 be an integer. Define ®,,(t) : S > R2”"*? by 


®,(t) = (Ui), Ui Bt), Ui 34), ..., UiB"@) 


= (cost, sint, cos 3f, sin 3f, cos 9f, sin 9r,..., cos(3’"t), sin(3r)). 


Parts | and 2 of the following result complete the proofs of Theorems 3.2 and 4.1, 
respectively. 


Theorem 4.4. Fix integers m > 2 ands > 2. Let Ay, be a set of 2(3” — 1) equally 
spaced points on S, and let Aj; be a cs subset of S obtained from A,, by replacing 
eachT € A, with a cluster of s points, all of which lie on a very small arc containing 
T. Then 


1. the polytope Py, := conv(®»,(Am)) is a cs 2-neighborly polytope of dimension 
2(m + 1) that has 2(3” — 1) vertices, 


A Tale of Centrally Symmetric Polytopes and Spheres 315 


2. the polytope Py,5 := ConV(®»(Am,s)) is a cs polytope of dimension 2(m + 1) 
that has N := 2s(3" — 1) vertices and more than (1 - =) (7) edges. 


The assumption m > 2 is only needed to guarantee that the dimension of P,, is 
exactly 2(m + 1) rather than < 2(m + 1). To prove that P,,, is cs 2-neighborly and 
Pm,.s has many edges for all m > 1, it is enough to show that each vertex of P,,,; is 
connected by an edge to all other vertices of P,,,; except possibly those coming from 
the opposite cluster. This can be done by induction on m. Since ®; = Up, the case 
of m = | is simply Corollary 4.3. For the inductive step, one relies on some standard 
facts about faces of polytopes along with an observation that the composition of 
®,, : R > R*°"+) with the projection of R?°"+) onto R” that forgets the first 
two coordinates is the curve tf > ®,,_;(3t), while the composition of ®,, with the 
projection of R?"+) onto R* that forgets all but the first four coordinates is the 
curve ®,; see [12, Section 3] for more details. Finally, to extend the construction 
of Theorem 4.4 to odd dimensions, consider the bipyramid over polytopes P,, and 
Pins : 

To summarize, 


2 
(1-3-4?) (5) < fmax(d, N; 1) < (1-2~“)- ai (4.1) 


Itis, however, worth pointing out that in view of the main result of [60], fmax (d, N; 1) 
might be closer to the right-hand side of Eq. (4.1) than to the left one. 

To extend the above discussion to higher-dimensional faces, we need to look more 
closely at the curve U;, and its convex hull. 

One crucial feature of the convex hull M, = conv(Z;(t) : t € S) C R** of the 

trigonometric moment curve is that it is k-neighborly, that is, for any p < k distinct 
points t),...,t) € S, the convex hull conv(Z(t1), ..., Z(tp)) is a face of My; see, 
for example, Chapter II of [11]. While the convex hull of U; is not k-neighborly, 
the following theorem, which is the main result of [13], shows that it is locally 
k-neighborly (cf. Theorem 4.2). 
Theorem 4.5. For every positive integer k there exists a number > < ax < ™ such 
that for an arbitrary openarc TY C S of length ax and arbitrary distinct p < k points 
ti,...,tp €T, the set conv(U;(t), ..., Ux(tp)) is a face of By := conv(U;(t) : t € 
S). 

The gap between the currently known upper and lower bounds on fmax (d, N; k — 
1) for k > 2 is much worse than the gap for k = 2 illustrated by Eq. (4.1). Indeed, 
we have: 


Theorem 4.6. Let 3 < k < d/2. Then 


2ok\ —4 N - - - 
(1-4 (mee) *) (7) smaxca, Wek) = (1-2-9 (7). 


The proof of the upper bound part of this theorem follows easily from the first part 
of Theorem 4.1 together with (1) the well-known perturbation trick that reduces the 
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situation to cs simplicial polytopes and (2) the standard double-counting argument 
that relates the number of edges to the number of (k — 1)-faces in any simplicial 
complex with N vertices, see [14, Proposition 2.2] for more details. 

The proof of the lower bound part can be found in [12, Section 4]. It utilizes a 
construction that is somewhat along the lines of the construction of Theorem 4.4, 
but quite a bit more involved: the desired polytope is obtained as the convex hull of 
a carefully chosen set of points on the curve Wy : S > R7*""*) defined by 


We m(t) = (Uxg(t), Ux, Bt), UE BH), 01944" D). 


(Note that ®,, is essentially W2,,,: in V2,,,(¢) every coordinate except for the first two 
and the last two shows up twice; to obtain ®,, from W2,,, simply leave one copy of 
each repeated coordinate.) To choose an appropriate set of points one uses the notion 
of ak-independent family of subsets of {1, 2, ... , m}andadeterministic construction 
from [25] of a large k-independent family. Finally, to show that the resulting polytope 
has many (k — 1)-faces, one relies heavily on the local neighborliness of the convex 
hull of U; discussed in Theorem4.5 along with some standard results on faces of 
polytopes. 

As the discussion above indicates, at present our understanding of the maximum 
possible number of faces of a given dimension that acs d-polytope with a given num- 
ber of vertices can have is rather limited even for d = 4. In particular, the following 
questions are wide open: 


fmax (d,N;1) 
ae 


Problem 4.7. Does the limit limy_s oo exist and, if so, what is its value? 


Or better yet, what is the actual value of fmax (d, N; 1)? Similarly, what is limy-+o 
nl NE) for 2 <k <d/2? Can we, at least, establish better lower and upper 


bounds than those given in Theorem4.6? 


In fact, it should be stressed that for d > 6 and N > 2(d + 2), we do not even 
know if in the class of cs d-polytopes with N vertices there is a polytope that simulta- 
neously maximizes all the face numbers. (For d = 4, 5, acs simplicial polytope that 
maximizes the number of edges automatically maximizes the rest of face numbers.) 


5 Toward an Upper Bound Theorem for cs Simplicial 
Spheres 


McMullen’s Upper Bound Theorem was extended by Stanley [73] to the class of all 
simplicial spheres. More precisely, Stanley proved that among all simplicial (d — 1)- 
spheres with n vertices, the boundary complex of the cyclic polytope Cqa(n) simul- 
taneously maximizes all the face numbers. (This result was extended even further to 
some classes of simplicial manifolds and even certain pseudomanifolds with isolated 
singularities, see [34, 58, 62]. In fact, already in 1964, Victor Klee [44] proved that 
the UBT holds for all (d — 1)-dimensional Eulerian complexes that have at least 
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O(d”) vertices. Whether the UBT holds for all Eulerian complexes remains an open 
question.) 

The situation with the face numbers of cs spheres versus face numbers of cs 
polytopes appears to be drastically different. On one hand, as we saw in Sects. 3 and 
4, acs 4-polytope P with N > 12 vertices cannot be 2-neighborly; furthermore, such 
a P has at most 2 . x edges. On the other hand, it is a result of Jockusch [36] that 


Theorem 5.1. For every even number N > 8, there exists a cs 2-neighborly sim- 


plicial sphere Jy of dimension 3 with N vertices. In particular, Jy has (%) -*- 


2 
= — N edges. 


Jockusch’s proof, see [36], is by inductive construction. The initial sphere, Js, is 
the boundary complex of the cross-polytope Cj with the involution ¢ induced by the 
map ¢(v) = —v on the vertex set of Cj. The cs sphere (Jy+2, @) is obtained from 
the cs sphere (Jy, ¢) by the following procedure: 


1. Finda subcomplex By in Jy such that (i) By is a 3-ball, (ii) By and (By) share 
no common facets, (iii) every vertex of Jy is a vertex of By, and (iv) every edge 
of By lies on the boundary of By, 1i.e., By has no interior vertices and no interior 
edges. 

2. Let vy+; and vy+2 be two new vertices. Cut out the interior of By from Jy, and 
cone the boundary of the resulting hole with vy,,. Similarly, cut out the interior 
of ¢(By) from Jy, and cone the boundary of the resulting hole with vy +2. 


Extending ¢ to Jy +2 by letting d(vy+41) = vy42 and d(vyi2) = Vy41 provides us 
with a free involution on Jy+2. Furthermore, choosing By in a way that is specified 
in Part (1) of the construction guarantees that the cs sphere Jy+2 is 2-neighborly 
(provided Jy was 2-neighborly). To allow for this inductive construction, an essential 
part of Jockusch’s proof is devoted to defining By in a way that ensures that the 
resulting simplicial sphere Jyy42 has a subcomplex By +2 with the same properties. 

The suspension of a simplicial complex A, &(A), is the join of A with the 0- 
dimensional sphere, that is, U(A) = {F, F U {uo}, F U{wo} : F € A}, where uo 
and wo are two new vertices. Observe that if A is cs, then © (A) is cs (with uo and wo 
being antipodes), and if A is a (d — 1)-sphere, then &(A) is ad-sphere; furthermore, 
if A is cs k-neighborly, then so is &(A). In particular, for every even N > 8, U(Jy) 
is acs 2-neighborly simplicial 4-sphere with N + 2 vertices. Jockusch’s results lead 
to the following problem on higher-dimensional cs spheres. 


Problem 5.2. Let d > 5 and let N > 2d be an even number. Is there a cs |d/2]- 
neighborly simplicial (d — 1)-sphere with N vertices? 


The boundary complex of C7 is the unique cs d-neighborly simplicial (d — 1)- 
sphere with 2d vertices. The boundary complex of conv(+e),..., eg, + ~ é;) 
is acs |d/2|-neighborly simplicial (d — 1)-sphere with 2(d + 1) vertices [51]; fur- 
thermore, in the case of an odd d, [16, Section 6.2] provides a construction of many 
cs |d/2]|-neighborly simplicial (d — 1)-spheres with 2(d + 1) vertices. Lutz [47, 
Chapter 4] found (by a computer search) several cs 3-neighborly simplicial 5-spheres 
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with 16 = 2(6 + 2) vertices; his examples have vertex-transitive cyclic symmetry. 
Suspensions of Lutz’s examples are cs 3-neighborly simplicial 6-spheres with 18 
vertices. For all other values of d and N, Problem 5.2 remains wide open. It is also 
worth pointing out that Pfeifle [64, Chapter 10] investigated possible neighborliness 
of cs star-shaped spheres—a class of objects that contains all boundary complexes 
of cs simplicial polytopes and is contained in the class of all cs simplicial spheres. In 
analogy with McMullen—Shephard’s result [51] about cs d-polytopes with 2(d + 2) 
vertices, he proved that for all even d > 4, and for all odd d > 11, there are no cs 
|d/2|-neighborly star-shaped spheres of dimension d — 1 with 2(d + 2) vertices. 

One of the main reasons for trying to resolve Problem 5.2 comes from the fol- 
lowing cs analog of the Upper Bound Theorem; this result is due to Adin [2] and 
Stanley (unpublished). 


Theorem 5.3. Among all cs simplicial (d — 1)-spheres with N vertices, acs |d/2]- 
neighborly (d — 1)-sphere with N vertices simultaneously maximizes all the face 
numbers, assuming such a sphere exists. 


We sketch the proof of Theorem 5.3 in the next section; as in the classic non-cs 
case, it relies on the theory of Stanley—Reisner rings and on the Dehn—Sommerville 
relations. 

To close this section, we posit a weaker and hence potentially more approachable 
version, of Problem5.2. Let A be a (d — 1)-dimensional simplicial complex and F 
a face of A. The link of F in AisIka(F):={GEeA: FUGEA, FONG=9}. 
(Thus, the link of F describes the local structure of A around F.) We say that A 
is pure if all facets of A have dimension d — 1; equivalently, for any face F € A, 
the link of F is (d — |F| — 1)-dimensional. Furthermore, we say that A is Eulerian 
if it is pure and the link of any face F € A (including the empty face) has the 
same Euler characteristic as S‘~!"'-!—the (d — |F| — 1)-dimensional sphere. In 
particular, the class of Eulerian complexes includes all simplicial spheres, all odd- 
dimensional simplicial manifolds as well as all even-dimensional manifolds whose 
Euler characteristic is two. Eulerian complexes were introduced by Victor Klee in 
[43]. 


Problem 5.4. Let d > 5 and let N > 2d be an even number. Is there a cs |d/2]- 
neighborly (d — 1)-dimensional Eulerian complex with N vertices? 


6 The Algebraic Side of the Story: Stanley—Reisner Rings 


We now switch to the algebraic side of the story and present a quick review of 
the major algebraic tool in the study of face numbers of simplicial complexes—the 
Stanley—Reisner ring. Along the way, we outline the proof of Theorem5.3 as well 
as prepare the ground for our discussion of the Lower Bound Theorems in the next 
section. The main reference to this material is Stanley’s book [77]. 

Let A be a simplicial complex with vertex set V and let K be an infinite field of 
an arbitrary characteristic. Consider the polynomial ring S := K[x, : v € V] with 
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one variable for each vertex in A. The Stanley—Reisner ideal of A is the following 
squarefree monomial ideal 


Inc= (Nii, Fm, : {vj, U2,..., ux} ¢ A). 


The Stanley—Reisner ring (or face ring) of A is the quotient K[A] := S/J,. Since 
I, is a monomial ideal, the quotient ring K[A] is graded by degree. The definition 
of 7, ensures that, as a K-vector space, each graded piece of K[A], denoted K[A],, 
has a natural basis of monomials whose supports correspond to faces of A. 

Stanley’s and Hochster’s insight (independently from each other) in defining this 
ring [35, 73] was that algebraic properties of K[A] reflect many combinatorial and 
topological properties of A. For instance, if A is (d — 1)-dimensional, then the Krull 
dimension of K[A] is d; in fact, the Hilbert series of K[A], i.e., Hilb(K[A], t) := 
Yo dimx K[A]; - t’, is given by 


“4 (A)ti a “(Ati d —14 
Hilb(K[A ho= DF (A) = Des (A)t"( ) 
d—1) (1 —1t)4 
The last equation leads to the following definition: if A is a (d — 1)-dimensional 
simplicial complex, then the h-vector of A, h(A) = (ho(A), 11 (A), ..., Ag(A)), is 
the vector whose entries satisfy aa h(A)ti = a fi-1(A)ti (1 — t)4-*; equiv- 
alently, 


d d 
DotA? = VF faye — (6.1) 


i=0 


In particular, hg(A) = 1,44(A) = fo(A) — d,andh2(A) = f(A) — (d— 1) fo(A) 
+ (). 

The following immediate consequences of Eq. (6.1) are worth mentioning: know- 
ing the f-numbers of A is equivalent to knowing its h-numbers. Moreover, since the 
jf -numbers are nonnegative integer combinations of the h-numbers, any upper/lower 
bounds on the h-numbers of A automatically imply upper/lower bounds on the f- 
numbers of A. 

Let A be a (d — 1)-dimensional simplicial complex. A sequence of linear forms, 
01, 02,..., 8g € Sis called a linear system of parameters (or |.s.o.p.) for K[A] if the 
ring K[A]/(©) is a finite-dimensional K-vector space; here (©) := (6), ..., 0a). It 
is well known that if K is an infinite field, then K[ A] admits an 1.s.o.p. An |.s.o.p. for 
K[A] is a regular sequence if 6;,) is a nonzero divisor on K[A]/(@, ..., 6;) for all 
0 <i <d. We say that A is Cohen—Macaulay (over K), or K-CM for short, if every 
1.s.o.p. for K[A] is a regular sequence. 

Assume now that A is K-CM, and 6),..., #y is an 1.s.o.p. for K[A]. Then 6; is a 
nonzero divisor on K[A], and so the following sequence of K-vector spaces 


0 > KIAli-1 —> KIA]: — (KIAI/@)), > 0 (6.2) 
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is exact (for all i > 0). Thus, dimg (K[A]/(@,)); = dimg K[A], — dimg K[A];-| 
for all i. Multiplying by t' and summing up the resulting equations, we obtain that 
Hilb(K[A]/(@)), t) = (1 — t) Hilb(K[A], r). Iterating this argument for 63, ..., 04 
leads to the following result due to Stanley [73, Section 4]. 


Theorem 6.1. Let A be a (d — 1)-dimensional K-CM complex and let 6,,..., 0a 
be an 1.s.o.p. for K[A]. Then Hilb(K[A]/(@), t) = (1 — t)4 Hilb(K[A], t) = ae 
h,(A)t'. (In Lae the h-numbers of CM complexes are nonnegative.) 


If A CT are simplicial complexes (say, on the same vertex set V), then I, D> Ip, 
and so there is a natural surjection K['] — K[A]. Furthermore, if dim A = dimT, 
then any l.s.o.p. 01,...,6¢ for K[P] is also an L.s.o.p. for K[A], and the induced 
graded homomenshismh KIT ]/(©) > K[A]/(©) is surjective. This observation 
together with Theorem 6.1 yields the following special case of [76, Theorem 2.1]: 


Corollary 6.2. Let A CT be simplicial complexes. Assume that both A and TV are 
K-CM and have the same dimension. Then h;(A) < h,() for all i. 


The reason CM complexes are relevant to our discussion is that, by a result of 
Reisner [66], A is a K-CM complex if and only if A is pure and for every face F of 
A (including the empty face), all but the top homology group (computed over K) of 
the link of F vanish. In particular, all simplicial spheres and balls are CM over any 
field; furthermore, the j-skeleton of a K-CM complex is K-CM for all /. 

Another very important result about simplicial spheres is Dehn—Sommerville 
relations established by Klee [43]: if A is a simplicial (d — 1)-sphere, then h;(A) = 
hg_j(A) for all 0 <i < d. (In fact, Klee showed that these relations hold for all 
Eulerian complexes.) 

We are now ready to close this section with a sketch of the proof of Stanley’s 
UBT and of its cs analog—Theorem 5.3. Following the custom, if P is a simplicial 
polytope, we denote by h(P) the h-vector of the boundary complex of P. 

Let A be a simplicial (d — 1)-sphere with vertex set V, |V| =n. Let V be the 
simplex on V, let T = Skelg_;(V), and let Cy(n) be the cyclic polytope. Then 
A CT are both CM complexes of the same dimension. Hence, by Corollary 6.2, 
h,(A) < h;() for all O <i < d. Furthermore, since Cg(n) is |d/2]-neighborly, 
OC,(n) and I have the same (|d/2] — 1)-skeleton, and so h;(T) = hj(Cqa(n)) for 
all 0 <i < d/2. (Indeed, h; is determined by fo, fi,..., fi-1.) We conclude that 
h,(A) < h;(Cqa(n)) for all 0 <i < d/2. Dehn—Sommerville relations, applied to A 
and 0C,(n), then yield that h;(A) < h;(Cq(n)) also holds for alld/2 <i < d. Thus, 
h,(A) < h;(Ca(n)) for all 0 < i < d, and the Upper Bound Theorem, asserting that 
F(A) < fj)(Ca(m)) for all 1 < j < d — 1, follows. 

Similarly, if A is a cs simplicial (d — 1)-sphere with N = 2m vertices, then A 
is a full-dimensional subcomplex of I := Skelg_;(OC*) (under any bijection from 
the vertex set of A to that of OC; that takes antipodal vertices to antipodal vertices). 
Further, if S is acs |d/2|-neighborly simplicial (d — 1)-sphere with N vertices, then 
Skel|g/2)-1(") = Skelja/2)-1(S). Applying the same argument as above to A, I, and 
S, we conclude that h;(A) < h;(S) for allO0 <i < d, and hence that f;(A) < f;(S) 
for all 1 < j < d — 1. This completes the proof of Theorem 5.3. 
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Note that whether a cs |d/2]|-neighborly simplicial (d — 1)-sphere with N = 2m 
vertices exists or not, the h-numbers (and hence also f-numbers) such a sphere 
would have are well defined: h;(S) = hg_;(S) = h;(Skelg_1(C;,)) for all i < d/2. 
Thus, independently of the existence of such a sphere, Theorem5.3 provides upper 
bounds on face numbers of cs simplicial (d — 1)-spheres with N vertices. However, 
if acs |d/2|-neighborly simplicial (d — 1)-sphere with N vertices exists, then these 
bounds are tight. 

Note also that the proof of Theorem 5.3 almost does not use the central symmetry 
assumption: instead, it relies on a much weaker condition, namely, on the existence 
of a free involution ¢ : V(A) — V(A) on the vertex set of A such that {v, d(v)} is 
not an edge of A for all v € V(A). We refer our readers to [59] for more details on 
the h-vectors of simplicial spheres and manifolds possessing this weaker property 
and to [17] for a complete characterization of h-vectors of CM complexes with this 


property. 


7 The Generalized Lower Bound Theorem for cs Polytopes 


Our ultimate dream is to find a cs analog of the g-theorem for cs simplicial polytopes. 
While at the moment it is completely out of reach (we do not even have a plausible 
upper bound conjecture, let alone a complete characterization!), establishing the 
lower bound-type results is a necessary part of the program. To this end, in this 
section we discuss a cs analog of the Generalized Lower Bound Theorem for cs 
simplicial polytopes. We start by reviewing the classical Lower Bound Theorem 
(LBT, for short) and the Generalized Lower Bound Theorem (GLBT, for short). To 
state these results, we need a few definitions. 

A triangulation of a simplicial d-polytope P is a simplicial d-ball B whose 
boundary, OB, coincides with OP. A simplicial d-polytope P is called (r — 1)- 
stacked (for some | <r <d) if there exists a triangulation B of P such that 
Skelg_,(B) = Skelg_,;(OP), ie., all “new” faces of this triangulation have dimen- 
sion > d —r. Note that the simplices are the only 0-stacked polytopes, and that 
1-stacked polytopes—usually referred to as stacked polytopes—are polytopes that 
can be obtained from the d-simplex by repeatedly attaching (shallow) d-simplices 
along facets. In contrast with the cyclic polytopes, two stacked d-polytopes with n 
vertices may not have the same combinatorial type. However, they do have the same 
face numbers. 


Theorem 7.1. (LBT) Let P be a simplicial d-polytope with d => 4. Then h\(P) < 
hy(P), with equality if and only if P is stacked. 


Theorem 7.2. (GLBT) Let P be a simplicial d-polytope. Then 
1 = ho(P) < hi (P) < ho(P) <--- < Aja (P). 


Furthermore, h,(P.) = h,_\(P) forsomer < d/2 if and only if P is (r — 1)-stacked. 
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Since f;-1 is a nonnegative linear combination of ho, h1,..., 4;, one immediate 
corollary of Theorem 7.1 is that among all simplicial d-polytopes with n vertices, a 
stacked polytope has the smallest number of edges. In fact, Theorem 7.1 together 
with a well-known reduction due to McMullen, Perles, and Walkup implies that 
among all simplicial d-polytopes with n vertices, stacked polytopes simultaneously 
minimize all the face numbers, and that for d > 4, stacked polytopes are the only 
minimizers, see [37, Section 5]. 

The differences between consecutive h-numbers are known as the g-numbers: 
go(P) := 1 and g,(P) := h,(P) — h,-1(P) for 1 <r < d/2. Nonnegativity of g2 
for simplicial polytopes of dimension 4 and higher was established by Barnette [10], 
while Billera and Lee [15] proved that the equality g.(P) = 0 (for d > 4) holds if 
and only if P is stacked. The first part of the GLBT was proved by Stanley [74] (a 
more elementary proof of this part is due to McMullen [49, 50]). The second part 
was conjectured by McMullen and Walkup, [52] and proved only recently by Murai 
and Nevo [55]. 

Since the d-dimensional cross-polytope has the minimal number of faces among 
all cs simplicial d-polytopes and since h, (C7) = (‘) > (?,) =h,_\(Cj) for all 1 < 
r <d/2, one expects that for this class of polytopes, the inequalities g,(P) > 0 of 
the GLBT can be considerably strengthened. Indeed, the following result of Stanley 
[75] provides a cs version of the inequality part of the GLBT. This result settled an 
unpublished conjecture by Bjérner. 


Theorem 7.3. Let P be acs simplicial d-polytope. Then 


gr(P) = (‘) - ( ‘ ) = 91C)) joralrsd/2. 
r r—-1l 


Stanley’s proof of the first part of Theorem7.2 is based on the theory of toric 
varieties associated with polytopes. Here is a very rough sketch of the argument. 
If P C R® is a simplicial d-polytope, then a slight perturbation of the vertices 
of P does not change its combinatorial type, and so we may assume without 
loss of generality that P has rational vertices; further, by translating P, we may 
also assume that the origin is in the interior of P. Let V be the vertex set of 


P. For each v € V, let (vj, v2,..., vg) denote the coordinates of v in R%, and 
define 0; := ea vjXy € R[x, : v € V] for j = 1,2,...,d. Consider R[OP]— 
the Stanley—Reisner ring of the boundary complex of P over R. Then 6),..., 9 


is an l.s.o.p. of R[OP] (this follows, for instance, from [77, Theorem III.2.4]), and 
so by Theorem 6.1, dimg (R[OP]/(©)); = h;(P). On the other hand, R[OP]/(©) is 
isomorphic to the singular cohomology ring of the toric variety X p corresponding to 
P (see [20, Theorem 10.8] and [26, Section 5.2] for more details). Letw = >, ev Xv: 
The toric variety X p is known to satisfy the hard Lefschetz theorem, which implies 
that for alli < d/2, the multiplication map -w : (RI[OP]/(©));_; > (RIOP]/(©)), 
is injective, and hence that h;_;(P) < h;(P) for all i < d/2, as desired. 

Extending Theorem 7.2 to all simplicial spheres is a major open problem in the 
field: it is a part of the celebrated g-conjecture. The main obstacle is that all known 
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proofs of the g-theorem for simplicial polytopes, see [49, 50, 74], rely heavily on such 
tools as toric varieties and the hard Lefschetz theorem or polytopal algebras and the 
Hodge—Riemann—Minkowski quadratic inequalities between mixed volumes, that is, 
tools that are (at least at present) available only for convex polytopes. 

To prove Theorem 7.3, a few additional ideas are needed. Let P be acs simplicial 
d-polytope or, more generally, let (A, ¢) be acs CM complex over R with vertex 
set V. Then the involution ¢ induces the map o : x, — x,y) on the set of variables 
of S = R[x, : v € V], which, in turn, extends to a unique automorphism oc on S. 
Furthermore, since 0(/,) = I,, the map o gives rise to an automorphism on R[A], 
which we also denote by o. The main insight of [75] is that this allows us to equip 
R[A] with a finer grading by N x Z/2Z: indeed, define 


RIAJ@+n = {f € RIA] : o(f) = f} and REA]@—y = {f € RIA] : o(f) = — fh. 


Then R[A]; = R[A]@,41) ® R[A](@,—1) as vector spaces while R[A](i,<,) - RIA](j,c) 
C RI[Alg+j,e,¢) for all i, 7 ¢ N and €,, €2 € {£1}. It is also not hard to see that 
dimer R[A]@,+41 => dimr R[A]@,-1 => + dime R[A]; for all i = 1, and that 
R[A]o+n = R[A]o = R. 

One can use the Kind—Kleinschmidt criterion [77, Theorem III.2.4] to show that 
R[A] has an Ls.0.p. 0),..., 0 such that 6; € R[A]q,-1) for all j. (For instance, 
for a cs simplicial polytope P and A = OP, the |.s.o.p. described in the proof of 
Theorem 7.2 does the job.) Analyzing the exact sequences as in the proof of Theorem 
6.1, see Eq. (6.2), but using our finer grading and, in particular, that -0; maps the 
degree (i — 1, +1) pieces of R[A]/(@1, ..., 6;—-1) to the degree (7, 1) pieces, then 
yields that 


dimg(R[A]/(®))a—1) = oe 5 (ia) 2 (‘)) for allO0 <i <d; (7.1) 
Ll 


see [75] for more details. This establishes the following result of Stanley [75]: 


Theorem 7.4. Let A be acs R-CM complex of dimension d — 1. Then h;(A) => (“) 
for alli. 


We now consider the case of A = OP, where P is a cs simplicial d-polytope 
(with rational vertices), and 6, ..., 04 are defined using the coordinates of vertices 
of P as in the sketch of the proof of Theorem 7.2; in particular, 0; € R[A](,-1). 
Observe thatw = )° cy X» € R[Ala, cere that -w maps (R[A]/(®)) @—1,-1) 
to (R[A]/(®))@,-1). As the map -w: A]/(©));_; @ (RIA]/(©)); is injective 
for all i < d/2, its restriction to (R[A ee ar is also injective. This, along 
with Eq. (7.1), completes the proof of Theorem7.3. It is worth pointing out that 
Theorem 7.3 was extended by Adin [3, 4] to the classes of all rational simplicial 
polytopes with a fixed-point-free linear symmetry of prime and prime-power orders. 
Furthermore, A’Campo-Neuen [1] extended Theorem 7.3 to toric g-numbers of all 
cs polytopes (including non-rational non-simplicial ones). 
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The Murai—Nevo proof [55] of the equality case of Theorem 7.2 involves a beau- 
tiful blend of tools such as Alexander duality, Stanley—Reisner rings, the Cohen— 
Macaulay property, as well as generic initial ideals, and, in particular, Green’s crys- 
talization principle [30, Proposition 2.28]. A different proof of both parts of Theorem 
7.2, including a sharper version of the inequality part, was very recently obtained by 
Adiprasito [5, Cor. 6.5 and §7]. His proof relies on Lee’s generalized stress spaces 
[45]. 

When does equality hold in Theorem7.3? That is, for a fixed r < d/2, is there 
a characterization of cs simplicial d-polytopes with g, = (‘) - (ee ayy In a recent 
paper [41], Klee, Nevo, Novik, and Zheng provide such a characterization in the 
r = 2 case and a conjectural characterization in the r > 2 case. Both statements 
strongly parallel the equality cases of Theorems 7.1 and 7.2. If P is a cs simplicial 
d-polytope with d > 4, then we can apply to P the symmetric stacking operation: 
this operation repeatedly attaches (shallow) simplices along antipodal pairs of facets. 
Note that symmetric stacking preserves both central symmetry and gz. 


Theorem 7.5. Let P be acs simplicial d-polytope with d > 4. Then go(P) = ‘@ - 
d if and only if P is obtained from Ci by symmetric stacking. 


Conjecture 7.6. Let P be acs simplicial d-polytope, and assume that g,(P) = 
() a (= ) for some 3 <r < |d/2]. Then there exists a unique polytopal complex C 
in R¢ with the following properties: (i) one of the faces of C is the cross-polytope C%, 
all other faces of C are simplices that come in antipodal pairs, (ii) C is a “cellulation” 
of P, that is, Ucee C = P, and (iii) each element C € C of dimension < d —r isa 
face of P. 


We mention that [41, Conjecture 8.6] provides a more detailed version of the 
above conjecture, and we refer our readers to Ziegler’s book [84, Section 8.1] for the 
definition of a polytopal complex. Conjecture 7.6, if true, would imply the following 
weaker statement that is also wide open at present. (The r = 2 case does hold; this 
is immediate from Theorem 7.5.) 


Conjecture 7.7. Let P be acs simplicial d-polytope. If g-(P) = g,(Cj) for some 
3<r <|d/2| —1, then g-41(P) = gr+1(C}). 


One consequence of Theorems 7.3 and 7.5, along with the fact that the f-numbers 
of spheres are nonnegative linear combinations of the g-numbers, is that among all 
cs simplicial d-polytopes with n vertices, a polytope obtained from C7, by symmetric 
stacking simultaneously minimizes all the face numbers; furthermore, if d > 4, then 
such polytopes are the only minimizers. 

While we will not discuss here the proof of Theorem 7.5, it is worth pointing out 
that it relies on the rigidity theory of frameworks (some ingredients of this theory 
are briefly outlined in the next section), and, in particular, on (1) the theorem of 
Whiteley [82] asserting that the graph of a simplicial d-polytope (d > 3) with its 
natural embedding in R¢ is infinitesimally rigid, and on (2) the fact that if P is 
acs simplicial d-polytope with go.(P) = G )- d, then all stresses on P must be 
symmetric (i.e., assign the same weight to each edge and its antipode.) This latter 
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observation follows from work of Stanley [75] and Lee [45], and also from more 
recent work of Sanyal, Werner, and Ziegler [68, Theorem 2.1]. 

To summarize: in contrast with the upper bound-type results, an analog of the 
GLBT for cs simplicial polytopes (at least its inequality part developed in Theo- 
rem7.3) is a very well-understood part of the story. Furthermore, in the case of the 
LBT we even have a characterization of the minimizers (see Theorem7.5). The 
part that is still missing is a characterization of cs simplicial d-polytopes with 
gr = (“) — (.4,) for 3 <r < |d/2]. Conjecture 7.6 proposes such a characteriza- 
tion. 


8 The Lower Bound Conjecture for cs Spheres and 
Manifolds 


The final part of our story concerns a conjectural analog of the LBT for cs spheres 
(manifolds and even normal pseudomanifolds)—a necessary step in our quest for a 
cs analog of the g-conjecture. To this end, it is worth recalling that in the world of 
simplicial complexes without a symmetry assumption, where at present the GLBT 
(Theorem 7.2) is only known to hold for the class of simplicial polytopes, works of 
Walkup [80], Barnette [9], Kalai [37], Fogelsanger [24], and Tay [78] show that the 
LBT (Theorem 7.1) holds much more generally: 


Theorem 8.1. Let A be a simplicial complex of dimension d — 1 > 3. Assume fur- 
ther that A is a connected simplicial manifold (or even a normal pseudomanifold). 
Then g2(A) = 0, with equality if and only if A is the boundary complex of a stacked 
polytope. 


In view of this result, we strongly suspect that Stanley’s inequality on the go- 
number of cs polytopes (see Theorem 7.3) as well as the characterization of the 
minimizers given in Theorem 7.5 continue to hold in the generality of cs simplicial 
spheres or perhaps even cs normal pseudomanifolds. This conjecture is however wide 
open at present. 


Conjecture 8.2. Let A be acs simplicial complex of dimension d — | > 3. Assume 
further that A is a simplicial sphere (or a connected simplicial manifold or even a 
normal pseudomanifold). Then g2(A) = (‘) — d. Furthermore, equality holds if and 
only if A is the boundary complex of a cs d-polytope obtained from the cross-polytope 
Ci by symmetric stacking. 


In the rest of this short section, we discuss one potential approach to attacking this 
conjecture: via the rigidity theory of frameworks. The (now wide) use of this theory 
in the study of face numbers of simplicial complexes was pioneered by Kalai in 
his celebrated proof of Theorem 8.1 for simplicial manifolds [37]. Below we review 
several results and definitions pertaining to this fascinating theory. We refer our 
readers to Asimow and Roth [6, 7] for a friendly introduction to this subject. 
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Let G = (V, E) be a finite graph. A map p: V > R¢ is called a d-embedding 
of G if aff{p(v) : v € V} = R*. The graph G, together with a d-embedding p, is 
called a d-framework. An infinitesimal motion of a d-framework (G, p) is a map 
pu: V(G) — R¢ such that for any edge {u, v} in G, 


‘| (p(u) + tu(u)) — (p(v) + tu(v)) I’ = 0. 


t=0 


d 
dt 


An infinitesimal motion jz of (G, p) is trivial if 4 al (p(u) + tu(u)) — (p(v) + 
t= 


tu(v)) | es 0 holds for every two vertices u, v of G. (Trivial infinitesimal motions 
correspond to a start of an isometric motion of R@.) We say that a d-framework (G, p) 
is infinitesimally rigid if every infinitesimal motion ju of (G, p) is trivial. 

A stress on a d-framework (G, p) is an assignment of weights w= (we : e€ 
E(G)) to the edges of G such that for each vertex v equilibrium holds: 


Y uni (P) = pw) = 0. 


u: {u,v}EE(G) 


We denote the space of all stresses on (G, p) by S(G, p). The stresses on (G, p) 
correspond to the elements in the kernel of the transpose of a certain f;(G) x dfo(G) 
matrix Rig(G, p) known as the rigidity matrix of (G, p). 

The relevance of rigidity theory to the Lower Bound Theorem is explained by 
the following fundamental fact that is an easy consequence of the Implicit Function 
Theorem (see [6, 7]). 


Theorem 8.3. Let (G, p) be a d-framework. Then the following statements are 
equivalent: 


e (G, p) is infinitesimally rigid; 
e rank Rig(G, p) = dfo(G) — (a8 
o dime S(G, p) = fi(G) — dfo(G) + (“4'). 


A graph G is called generically d-rigid if there exists a d-embedding p of G 
such that (G, ) is infinitesimally rigid; in this case, the set of infinitesimally rigid 
d-embeddings of G is an open dense subset of the set of all d-embeddings. Recall that 
if A is a (d — 1)-dimensional simplicial complex, then g2(A) = h2(A) — hy(A) = 
fi(A) — dfo(A) + (“4'). The last condition of Theorem 8.3 then implies that if the 
graph (i.e., 1-skeleton) of A is generically d-rigid, then g2(A) > 0. 

Two basic but very useful results in rigidity theory are the gluing lemma ([7, 
Theorem 2] and [83, Lemma 11.1.9]) and the cone lemma [81]. The gluing lemma 
asserts that if two graphs G; and Gz are generically d-rigid and share at least d 
vertices, then their union G, U Gz is also generically d-rigid, while the cone lemma 
posits that G is generically d-rigid if and only if the graph of the cone over G 
is generically (d + 1)-rigid. Additionally, by Gluck’s result [29], the graph of any 
simplicial 2-sphere is generically 3-rigid. Now, if A is a simplicial (d — 1)-manifold 
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(with d > 4) and F is a (d — 4)-face of A, then the link of F in A is a simplicial 
2-sphere and hence has a generically 3-rigid graph. This observation, along with the 
gluing and cone lemmas, allowed Kalai [37] to prove by induction on d > 4 that the 
graph of any simplicial (d — 1)-manifold is generically d-rigid and thus to establish 
the inequality part of Theorem 8.1 for all simplicial manifolds. 

The relationship of infinitesimal rigidity to the Stanley—Reisner ring was worked 
out in [45]. Specifically, let A be a (d — 1)-dimensional simplicial complex with 
vertex set V and let p : V — R“ be ad-embedding of the graph of A, G(A). Define 
d linear forms in R[x, : v € V] by 6; := )oycy p(v) jx for j = 1,..., d, where 
p(v); denotes the jth coordinate of p(v) € IR¢, and let 0741 := Dey Xe (el. Sect.7, 
and especially the sketch of the proof of Theorem7.2). Theorem 10 of [45] implies 
that if (G(A), p) is infinitesimally rigid, then dimp(R[A]/(, ..., 0a, 9a41))2 = 
dimp S(G(A), p) = g2(A). Equivalently, if (G(A), p) is infinitesimally rigid, then 
6:41 : (R[AJ/(@1,.-.,6;))1 > (RIAI/(@,...,9;))2 is an injection for all 
O<ik<d. 

Assume now that A is centrally symmetric. If there is ad-embedding p : V(A) > 
IR¢ that respects symmetry and such that (G(A), p) is infinitesimally rigid, then the 
previous paragraph along with the N x Z/2Z-grading of R[A] and relevant com- 
putations of Sect.7 (e.g., Eq. (7.1)) shows that g.(A) > () — d. (This also follows 
from the argument in [68, Section 2].) In particular, the following conjecture, if true, 
would imply the inequality part of Conjecture 8.2. 


Conjecture 8.4. Let A be a (d — 1)-dimensional cs simplicial complex with an 
involution @. Assume further that A is a simplicial sphere (or a connected simplicial 
manifold or even a normal pseudomanifold) and that d — | => 3. Then there exists 
a d-embedding p: V(A) — R¢ that respects symmetry, i.e., p(o(v)) = —p(v) for 
each vertex v of A, and such that (G(A), p) is infinitesimally rigid. 


For instance, if A is the boundary complex of a cs simplicial d-polytope P C R¢, 
then by Whiteley’s theorem [82], the natural d-embedding of the graph of P qualifies 
for ¢ as in Conjecture 8.4. 

One of the reasons Conjecture 8.4 appears to be hard in the general case is that 
the links of cs complexes are usually not centrally symmetric, and so starting with 
cs 2-spheres as the base case (and proceeding by induction via the cone and gluing 
lemmas) does not work. Conjecture 8.4 of [41] proposes a statement about rigidity of 
graphs of (non-cs) simplicial 2-spheres that, if true, will provide an appropriate base 
case, and imply Conjecture 8.4. In any case, as a step in our quest for acs analog of the 
g-conjecture, it would be extremely interesting to shed any light on Conjectures 8.2 
and 8.4 as well as to attempt to strengthen the inequality of Conjecture 8.2 in the 
spirit of results from [54, Theorem 5.3(i)] and [56]. Such strengthened inequalities 
would provide lower bounds on gz of a cs simplicial manifold (or even a normal 
pseudomanifold) A in terms of the first homology or perhaps even in terms of the 
fundamental group of A and/or of the Z/2Z-quotient of A. 
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9 Concluding Remarks 


The recent decades made the theory of face numbers into a very active and a rather 
large field. Consequently, there are quite a few topics we glossed over or omitted in 
this paper. Among them are the face numbers of general (not necessarily simplicial) 
cs polytopes. However, one of the conjectures about general cs polytopes we cannot 
avoid mentioning is Kalai’s “3¢-conjecture” [39, Conjecture A]: it posits that the 
total number of faces in a cs d-polytope (including the empty face but excluding the 
polytope itself) is at least 3“. If the conjecture is true, there are multiple minimizers: 
the class of polytopes with exactly 3¢ faces includes at least all the Hanner polytopes. 
(If Mahler’s conjecture holds, then Hanner polytopes are the minimizers of Mahler 
volume among all the cs convex bodies.) At present the 3¢-conjecture is known to 
hold for all cs simplicial polytopes (this is an immediate corollary of Theorem 7.4) 
and, by duality, all simple polytopes, as well as for all at most 4-dimensional cs 
polytopes [68]. The conjecture is wide open in all other cases. 

We have also only barely touched on the face numbers of cs simplicial manifolds 
and instead concentrated on the face numbers of cs simplicial polytopes and spheres. 
Papers [59], [61, Section 4], and [42] contain some results pertaining to the Upper 
Bound Theorem for cs simplicial manifolds as well as to Sparla’s conjecture [71, 
Conjecture 4.12], [72] on the Euler characteristic of even-dimensional cs simplicial 
manifolds. 

While we had to skip quite a few of the topics and could not possibly do justice 
to all of the existing methods, we hope we have conveyed at least some of the 
essence and beauty of this fascinating subject! We are very much looking forward to 
progress on the many existing as well as yet unstated problems about cs polytopes 
and simplicial complexes, and to new interactions between combinatorics, discrete 
geometry, commutative algebra, and geometric analysis that will lead to this progress! 


Acknowledgements I am grateful to Steve Klee, Connor Sawaske, Hailun Zheng, Giinter Ziegler, 
and the anonymous referee for numerous comments on the preliminary version of this paper. 
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Anna Puskas 


Abstract Weyl group multiple Dirichlet series and metaplectic Whittaker functions 
can be described in terms of crystal graphs. We present crystals as parameterized 
by Littelmann patterns, and we give a survey of purely combinatorial constructions 
of prime power coefficients of Weyl group multiple Dirichlet series and metaplec- 
tic Whittaker functions using the language of crystal graphs. We explore how the 
branching structure of crystals manifests in these constructions, and how it allows 
access to some intricate objects in number theory and related open questions using 
tools of algebraic combinatorics. 


1 Introduction 


Crystal graphs are combinatorial objects appearing in the representation theory of 
semisimple Lie algebras. To an irreducible representation of a semisimple Lie algebra 
g, one may associate a crystal graph C. The vertices of this graph are in bijection 
with a weight basis of the representation, and the edges are colored by a set of simple 
roots of g. 

Crystals were first studied in connection with the representation theory of the 
quantized universal enveloping algebra. However, in this chapter it is their structure 
as a colored (directed) graph and their symmetries related to the Wey] group of g that 
are of interest to us. Crystals turn out to be a valuable tool in constructing certain 
objects from number theory: coefficients of multiple Dirichlet series and metaplectic 
Whittaker functions. 

Interest in multiple Dirichlet series and metaplectic Whittaker functions is moti- 
vated by hard questions in analytic number theory, for example, the Lindelof Hypoth- 
esis, and the study of automorphic forms [20]. The relevant literature in number theory 
is extensive (see Sect. 1.2). However, since these objects have constructions that are 
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almost purely combinatorial in nature, their study can be approached using tools of 
algebraic combinatorics. 

In this chapter, we are interested in this approach. Our main goal is to present 
combinatorial constructions of metaplectic Whittaker functions and coefficients of 
multiple Dirichlet series corresponding to root systems of the four infinite families 
of Cartan types. To do so, we use the language of Littelmann patterns. We highlight 
how the branching structure of crystals is apparent in the constructions and indicate 
how this aspect turns out to be significant in the study of the related questions from 
number theory. 

Before giving an overview of the structure of the chapter, we say a few more 
words on the relevant objects. 

Crystal graphs can be parameterized (without referring to the representation the- 
ory of the quantum group) using a variety of combinatorial devices, such as the Lit- 
telmann path model, Gelfand—Tsetlin patterns, Lusztig’s parametrization [35, 36], 
tableaux of Lakshmibai and Sheshadri [32] or Kashiwara and Nakashima [31]. For 
a thorough introduction to the theory of crystals from a combinatorial perspective, 
the reader is encouraged to consult [18]. 

Here we present crystals in terms of Berenstein—Zelevinsky—Littelmann paths 
and Littelmann patterns [34]. Our reason for this choice is twofold. First, most of 
the constructions in number theory that we are concerned with were either originally 
given in this language, or are easily rephrased in such terms. Moreover, phrasing the 
constructions in terms of Littelmann patterns highlights the role of the branching 
structure of crystals (as well as the significance of some “nice elements” of the Wey] 
group) very well. 

A major hurdle any expository writing on this topic has to overcome is the inher- 
ent intricacy and volume of the theory of multiple Dirichlet series and metaplectic 
Whittaker functions. Since we wish to take a purely combinatorial approach, we 
largely try and circumvent this issue. Some background on multiple Dirichlet series 
and Whittaker functions (as well as on metaplectic groups) will be given in Sect. 2.4. 
For now we say that through their connection to an algebraic group over a local 
or global field, these objects from number theory are related to the representation 
theory of the underlying Lie algebra g. Their constructions involve producing, for 
a dominant weight A, a polynomial P,(x) in r variables, where r is the rank of 
the Lie algebra g. In Sect.2.4, we shall say more about how a Weyl group multiple 
Dirichlet series or a metaplectic Whittaker function gives rise to such a polynomial 
P). However, for most of the chapter we shall ignore details of this background and 
concern ourselves with producing a polynomial P) as a sum over a crystal graph. 
The “constructions” mentioned throughout the chapter refer to constructions of a 
polynomial P,, depending on the context, this may agree with a p-part of a Weyl 
group multiple Dirichlet series or certain values of a spherical Whittaker function. 

The combinatorial perspective of focusing our attention on the polynomials P) 
is, on the one hand, helpful when considering questions motivated by the analytic 
background. On the other hand, these objects are interesting in their own right. This 
is due to the fact that they can be thought of as deformations of highest weight 
characters. As a result, techniques of character theory come into play. As a method 
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to study these polynomials, it provides insight into the original analytical objects. 
Furthermore, it motivates further questions. 

To provide an example, we mention two aspects of the polynomials P, now. One 
is Weyl symmetry: the polynomials P, inherit certain functional equations under the 
Weyl group corresponding to the underlying root system. Hence one may construct 
such a polynomial by (1) taking a sum over an object that is similarly symmetric, 
such as a highest weight crystal or (2) by taking an “average” over the Wey] group. 
(See Sect. 1.2 for relevant results in the literature.) Understanding the relationship 
between these two approaches is a large part of the motivation between studying the 
constructions combinatorially, and in the cases where the question is resolved, the 
branching structure of crystals turns out to play a significant role. 

We briefly explain the relevance of character theory. In the simplest special case, 
P) looks very similar to the deformation of a Schur polynomial; more generally, to 
the deformation of a Wey] character. (For P, a Whittaker function, this is a conse- 
quence of the Casselman—Shalika formula.) Hence one expects that the behavior of 
families of polynomials P) will be similar to the behavior of characters. On the one 
hand, this means that identities of Weyl characters may provide a useful tool of study. 
These come in a couple of different flavors. For example, the Weyl(-Kac) character 
formula produces a character. Branching rules describe the behavior of characters 
under restriction. Indeed, Tokuyama’s theorem (a deformation of the Wey] character 
formula) turns out to be key in investigating the relationship of the two approaches (1) 
and (2) mentioned above. Generalizing it to the polynomials P) requires understand- 
ing the branching properties of the P,. We shall elaborate on this point in Sects. 1.3 
and 1.4 below. 

On the other hand, one may ask if products of the P) satisfy some “enhanced” 
version of other character identities. For example, does a product of such polynomials 
satisfy “deformed” Pieri and Littelwood—Richardson rules? Question of this flavor 
may be investigated using any description of these objects. We shall see that the P, 
can be defined in terms of the combinatorial structure of a crystal and a few Gauss 
sums. Hence any question about them can be phrased in terms of the crystal structure 
and identities of Gauss sums. 

We give an overview of the structure of the chapter. 


1.1 Structure of the Paper 


In the remainder of this Introduction, we first give a brief review of results con- 
structing Weyl group multiple Dirichlet series or metaplectic Whittaker functions 
(Sect. 1.2). In Sect. 1.3, we explain how a theorem of Tokuyama is related to this 
topic, and how Demazure—Lusztig operators can be used to study, and extend the 
constructions discussed in this chapter to greater generality. Section 1.4 provides 
some further insight into the meaning and significance of branching. 

Littelmann patterns and their bijection with crystal elements are discussed in 
Sect.3. The constructions of Whittaker functions and prime power coefficients of 
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multiple Dirichlet series in terms of highest weight crystals are presented in Sect. 4, 
and the relationship of the constructions with the branching structure of crystals is 
highlighted in Sect. 5. 

Section 2 serves to present some preliminaries. We introduce notation (Sect. 2.1) 
and present Gauss sums, a necessary arithmetic ingredient to the constructions 
(Sect. 2.2). We then give a brief introduction to crystals and Berenstein—Zelevinsky— 
Littelmann paths (Sect.2.3). We also provide a little more insight into how coef- 
ficients of multiple Dirichlet series and Weyl group metaplectic Whittaker func- 
tions give rise to polynomials P(x), related to sums over highest weight crystals 
(Sect. 2.4). 


1.2 A Review of Literature 


We discuss the literature of constructions of multiple Dirichlet series and Whittaker 
functions. Our interest here is from the perspective of combinatorics. Hence we shall 
focus on the role of the branching structure of the crystals and the significance of 
special words in the Weyl group. For an insightful and thorough introduction to the 
topic from a number-theoretic perspective, the reader is encouraged to consult [17], 
the Introduction of the volume where many of the constructions discussed in this 
chapter were published. The role of this section is to provide this topic with a wider 
context; strictly speaking, it is not necessary for the understanding of any of the later 
parts. 

Brubaker, Bump, Chinta, Friedberg, and Hoffstein [12] introduced Weyl group 
multiple Dirichlet series (WMDS), series in several complex variables with functional 
equations governed by a finite Weyl group, corresponding to a root system © of finite 
type. As mentioned above, there are two separate approaches to how to associate a 
WMDS to a root system: by taking a sum over a crystal, or by Chinta and Gunnells 
[21], by averaging over the Wey] group. 

The authors of [12] conjecturally related WMDS to Whittaker coefficients of 
metaplectic Eisenstein series. This connection is of interest in that it allows one 
to prove functional equations and analytic continuation of the constructed series. 
Elucidating this connection motivates study of these objects as well. In the following 
paragraphs when we refer to a “conjectural description” of a WMDS as a sum over a 
crystal, we mean either that the constructed series is conjectured to be the Whittaker 
coefficient of a metaplectic Eisenstein series, or that it is conjectured to agree with a 
series constructed via the averaging method. 

We shall mention relevant results in all four infinite families of Cartan types; some 
of these results will be covered in more detail in Sect. 4. 

Brubaker, Bump, and Friedberg [15] describe the Fourier—Whittaker coefficients 
of Eisenstein series on a metaplectic cover of the general linear group as a Weyl 
group multiple Dirichlet series. They compute the prime power coefficients (p-parts) 
of these series in terms of the string parametrization of a crystal by Berenstein and 
Zelevinsky [4, 5] and Littelmann [34]. In [14], the same authors further explore the 
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combinatorics in the type A case. They give two separate constructions of the p- 
part in type A. These can be seen as corresponding to two different choices of nice 
decompositions of the long element of the Weyl group. The authors then prove that 
the two descriptions give the same p-parts through a subtle combinatorial argument. 
The equivalence of the two statements allows them to prove analytic continuation and 
functional equations for the emerging multiple Dirichlet series. In proving the equiva- 
lence, they observe the significance of some purely combinatorial phenomena—such 
as the Schiitzenberger involution. Their method provides an example of how to build 
p-parts of multiple Dirichlet series out of finite crystal data. 

Beineke, Brubaker, and Frechette [1, 2] give a definition fora WMDS in terms of 
statistics on a highest weight crystal of Cartan type C. They prove analytic continu- 
ation and functional equation of such series using a connection to Eisenstein series 
over odd orthogonal groups in the nonmetaplectic case, and conjecture a similar 
connection in general. 

Friedberg and Zhang [26] study Eisenstein series over metaplectic covers of odd 
orthogonal groups. They then describe the p-parts of the MDS that are the Whittaker 
coefficients of these series in terms of type C highest weight crystals. They in fact 
give two descriptions. The first one is only valid in the case of odd covers; this proves 
the conjectured connection in [2] above. The second is uniform in the degree n of the 
metaplectic cover, but the assignment of number-theoretic data to the combinatorial 
structure is more subtle. An interesting feature of their methods is that they are is 
inductive by rank. Furthermore, the proof of the agreement of two descriptions relies 
on the type A theory of [14]. 

As for type D, Chinta and Gunnells [22] give a conjectural construction of a 
Weyl group multiple Dirichlet series of type D. The p-part of a series is produced 
as a sum over a highest weight crystal associated to an irreducible representation of 
SO(2r). The contribution of a crystal element to the sum is described in terms of 
the corresponding Littelmann pattern. 

We also mention constructions of Whittaker functions. McNamara [39] consid- 
ers Whittaker functions on metaplectic covers of a simple algebraic group over a 
nonarchimedean local field. The Whittaker function is given as an integral (over the 
unipotent radical). Given a reduced decomposition of the long element in the Wey] 
group, one may break up the domain of integration into a set of cells. These cells are 
in a natural bijection with elements of an (infinite) crystal. By computing the integral 
on each cell, the Whittaker function is produced as a sum over a(n infinite) crystal 
structure. In type A, the resulting formula for the Whittaker function agrees with the 
formula given for local parts of a Weyl group multiple Dirichlet series by [15]. 

The averaging approach can also be used to construct metaplectic Whittaker func- 
tions as shown by Chinta and Offen [24] in the type A case, and McNamara [40] in 
general. Work of McNamara thereby provides a number-theoretic proof that the two 
methods (averaging and crystal constructions) produce the same local parts. 

From a combinatorial perspective, the formulas produced by the two separate 
approaches (averaging or sum over a crystal) are related in the nonmetaplectic case 
by a theorem of Tokuyama [45]. More generally, Demazure operators can be used 
to elucidate the connection between the two approaches [23, 44] combinatorially. 
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This relies heavily on the branching properties of crystals. This is explained in more 
detail in Sects. 1.3 and 1.4. 

The reason for the emergence of crystal bases in the study of these topics in itself 
warrants further exploration. Some results of this flavor exist both in the local and 
in the global setting. As mentioned above, the work of McNamara [39] expresses a 
Whittaker function as a sum over cells of the unipotent radical. The cell decompo- 
sition is then related to geometric realizations of the crystal in terms of Lusztig data 
[37] and Mirkovié—Vilonen cycles [7]. In the global setting, Brubaker and Friedberg 
[8] study Whittaker coefficients of metaplectic Eisenstein series induced from max- 
imal parabolics. They produce a formula for the Whittaker coefficient for a wide 
class of long words in the Weyl group by matching contributions with Lusztig data 
through MV polytopes considered by Kamnitzer [29]. 

In addition, highest weight crystals are not the only combinatorial device that 
is of use in constructing these number-theoretic objects. An other approach uses 
metaplectic ice [9, 10, 13]. 


1.3 Tokuyama’s Theorem 


In this section, we explain how the results above relate to a deformation of the Wey] 
character formula by Tokuyama [45], and how understanding the branching structure 
of crystals elucidates the relationship of the constructions. 

The constructions produce a polynomial P\(x) that satisfies certain functional 
equations under a Weyl group W. (Here we assume that the polynomial ring C[x] 
in r variables is identified with the group algebra of the weight lattice.) This is done 
either by taking a sum over a highest weight crystal C), or by taking an average over 
the Weyl group. We explain how both of these strategies results in a polynomial that 
is, roughly speaking, a deformation of a Wey] character. 

First let us consider the method of producing the polynomial P) by taking a sum 
over the crystal graph: 

Py(x) = )) Gx" (1) 


beCy 


The elements of the crystal C) are in bijection (via the weight map wt) with a weight 
basis of a representation of highest weight A. Note that if we had G(b) = 1 for every 
element of the crystal in (1), then the resulting sum would be the character 1 of this 
highest weight representation: 


1 
> 1-x"® — ym) = ia Yep w(x?) (2) 
bECy wew 


Here the right-hand side is the Wey] character formula, and A is a Wey] denominator. 
In general G(b) is more complicated, but in the simplest case we have that in fact 
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Py(x) = S° G(b)x™ = Ag - XA) (3) 
beECy 


where A, is a deformation of the Weyl denominator. 

Next let us consider the “averaging approach” to constructing the polynomial 
P)(x). This approach produces P(x) by an expression similar to the right-hand side 
of (2). However, the action of w on the monomial x**+? is replaced by the Chinta— 
Gunnells action [21], which depends on the metaplectic degree n. In the special case 
of n = 1, this construction results in the expression A, - y(x) as well. 

The statement that the two approaches to constructing P) give the same result 
can thus be phrased as a combinatorial identity, a deformation of the Wey] character 
formula. In the nonmetaplectic case, this is the second equality in (3), and this identity 
is a theorem by Tokuyama [45]. 

When n > 1, then understanding the relationship between the two constructions 
of P, combinatorially amounts to proving a metaplectic analogue of Tokuyama’s 
theorem. In the type A case, this was done by the author in [44] using metaplectic 
Demazure—Lusztig operators defined in [23]. 

We mention that analogues of Tokuyama’s theorem for root systems of other 
types have been given by Hamel and King [28] (for type B) and Friedlander, Gaudet, 
and Gunnells [27] (in type G2). Note also that the agreement between the rele- 
vant constructions in the type A case follows from work of McNamara as indicated 
above. However, treating these sides combinatorially via Demazure—Lusztig opera- 
tors allows one to understand how the constructions can be extended to more general 
settings, for example, from the finite dimensional to the general Kac—Moody set- 
ting [42], or, from Whittaker functions to the constructions of Iwahori—Whittaker 
functions [43]. 

The proof of the metaplectic analogue of Tokuyama’s theorem in [44] relies 
heavily on the type A crystal construction “respecting” the branching structure of 
the highest weight crystal. We explain this in more detail next. 


1.4 Motivation: Demazure-Lusztig Operators and the 
Branching Structure 


As seen above, understanding the combinatorial relationship between different con- 
structions of the polynomial P, (which may be a Whittaker function or the prime part 
of a WMDS) amounts to proving a metaplectic analogue of Tokuyama’s theorem. 

Using Demazure—Lusztig operators, one may phrase a more general identity, 
corresponding to elements w of the Weyl group, and any metaplectic degree n. The 
more general identity [44, Theorem 1.] is of the form: 


(= n) Y= > G(b)x" (4) 


u<w bec) 
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Here the expression on the left-hand side can be thought of as the general form of the 
expression produced by the averaging method (by results in [23]) and the right-hand 
side is a sum over a Demazure crystal. The “metaplectic analogue of Tokuyama’s 
theorem” (in type A) is the special case of this statement corresponding to w being 
the long element of the Weyl group. This more general statement has the advantage 
that it can be proven “one simple reflection at a time,” i.e., by induction on the length 
£(w) of the Weyl group element. 

The fact that the construction of P, as a sum over a highest weight crystal respects 
the branching structure of the crystal is crucial to the proof. We explain what we mean 
by this below. We shall return to this discussion in more detail in Sect.5 equipped 
with the necessary background. 

The crystal C) is graph, whose edges are labeled by simple roots a; (1 <i <r) 
of an underlying Lie algebra or rank r. When the edges labeled by a, are omitted, 
the remaining graph is a disjoint union of crystals C,,, corresponding to a Lie algebra 
of the same Cartan type as g, but rank r — 1: 


Cy =u, (5) 


A crystal element b € Cy has a contribution G(b) = G)(b) in the sum (1). The 
element b belongs to exactly one of the rank r — 1 crystals C,, C Cy. The element 
b has a contribution in the analogous construction of P,,(x). By the constructions 
respecting the branching structure, we mean that we have 


Gy(b) = g(u) - Gy), (6) 


where the factor g() is the same for every element b € C,, C Cj. It follows that Py 
can be written as an expression of polynomials P,, corresponding to the weights jz 
of the decomposition (5) above. 

This means that statements about these crystal constructions are amenable to 
proof by induction on rank. The parametrization of crystal elements by Littelmann 
patterns highlights the branching structure of crystals. We encourage the reader to 
keep the branching structure in mind while reading through the sections covering the 
ingredients of the constructions. 


2 Preliminaries 


Before describing the constructions mentioned above, we cover a few preliminaries. 
The constructions in Sect.4 have two main ingredients: a set of root data and an 
arithmetic ingredient in the form of certain Gauss sums. We introduce notation and 
the necessary ingredients below. 
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2.1 Notation 


Throughout the paper, ® shall denote a root system of rank r, with ®+ (respectively 
®~) being the set of positive (respectively, negative) roots. Let A = {a ,..., a@-} be 
a set of simple roots in ®, and let us write p = 5 > sco+ 6 for the Weyl vector. Of 
particular interest are the root systems of Cartan types A, B, C and D. We give [6] as 
a general reference on root systems. Note however that when discussing Littelmann 
patterns [34], our numbering of the simple roots agrees with that of Littelmann and 
hence differs from that of Bourbaki. (For example in type D, the simple roots a, and 
Q2 are orthogonal.) 

Letus write 0), ..., 0, for the set of simple reflections corresponding to the simple 
roots; a; is the reflection through the hyperplane perpendicular to a;. The Weyl group 
W is generated by the simple reflections o; (1 < i < r). Every element w € W can 
be written as a product w = gj, ---0;,. We call this a reduced decomposition and 
w = [i,..., 7%] areduced word if k is minimal and k = €(w), the length of w. The 
Weyl group has a unique longest element wo € W. The parameterization of highest 
weight crystals by Littelmann patterns given in Sect. 3 depends on a choice of a nice 
decomposition W, of the long element. 

The Weyl group permutes the elements of &. Let ®(w) = w7!(®-)N ®*, then 
£(w) = |®(w)| and ®(wo) = ®*. We shall denote the weight lattice corresponding 
to ® by A, and the fundamental weights corresponding to the basis A by a, ..., @,. 
The constructions we are concerned with produce polynomials in C[A]. The Weyl 
group has a natural action on A, and hence on C[A], it thus makes sense to talk about 
the resulting polynomials being symmetric (or having functional equations) under 
the Weyl group. 


2.2. Gauss Sums 


Next we introduce notation for the arithmetic ingredients of the polynomials con- 
structed in Sect. 4. The contribution of a crystal element is given by nth-order Gauss 
sums, where n is a positive integer. (In applications, n is the degree of the metaplectic 
cover.) Recall that (form = 2), one may take the quadratic Gauss sum 


£ k ka2zi 
G(a) = > (=) er (7) 


k=0 
where (4) is the Legendre symbol; i.e., it is 1 if k is a square modulo p and —1 


otherwise. The Gauss sums appearing in the construction are generalizations of the 
one in (7). The Legendre symbol is replaced by an nth power residue symbol (a 


ka2ri 
multiplicative character), and e ’ is replaced by an additive character. 
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To state the definition of the Gauss sums g(a)(=g;(a)) and h(a)(=h;(a)), we 
introduce some more notation. We use the language of local and global fields, and 
provide examples (see [41] for reference). The Gauss sums g(a) and h(a) are func- 
tions that depend on the residue of a modulo n. For the purpose of understanding the 
constructions and their relationship to the branching structure of crystals, the values 
of these functions are not crucial. 

Following [11], let F be a global field; the reader may choose to simply think of 
Qas an example. For a place v of F, one may take the completion F,,. (For example, 
the completion Q, of p-adic numbers at any finite prime p, or the completion R 
at the infinite place.) Let O, denote the set of integers (e.g., Zp,» C Q, or ZC R). 
Let S be a finite set of places of F, and let Os denote the set of S-integers x € K 
such that x € O, for every v ¢ S. (For S = {o00, 2}, the set Os C Q is the set of 
rational numbers with only 2 in the denominator.) For a sufficiently large S, Os is 
a principal ideal domain. Let F's = [],,-5 Fv, Os embeds into Fs diagonally. Let 
be a character of Fs trivial on Os and no larger fractional ideal. Let (:), denote the 
nth-order residue symbol and ¢ a positive integer. We define 


gacd= >> (5) o(), (8) 


d modc 


The constructions in Sect.4 will involve special values of g;(a, c). We shall have 
t = | ort = 2 be the length of a simple root (i.e., t = | in the simply laced cases and 
t = | ort = 2 intype B or type C), and we shall have fixed a prime p. Then we set 


|(Os/pOs)*| if t~'n | a 


g(a) = 9:(p*"', p*); and h,(a) = g:(p", p®) = {4 ift—n}a (9) 


In the remainder of the paper, we use the notation g = |Os/pOs| for the order of 
a residue field. 

The polynomials P) that we shall define in Sect. 4 are given as asum over a crystal. 
Each term is determined via combinatorial data coming from the parameterization 
of the corresponding crystal element via a method that makes use of the above Gauss 
sums. For example in Cartan type A, the Gauss sums take n values (indexed by the 
residue classes modulo n). Consequently, any statement about these polynomials can 
be phrased entirely in terms of the structure of the highest weight crystal and identities 
of these nth-order Gauss sums. Such identities are rare. In addition to the identity 
expressing the relationship of Gauss sums corresponding to conjugate characters, 
one has the Hasse—Davenport relations [25]. By work of Yamamoto [46], these are 
essentially the only multiplicative identities of these Gauss sums. We thank one of 
the referees for pointing this out. 
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2.3 Highest Weight Crystals and Littelmann’s Cone 


Given an irreducible (finite) root system ® and a dominant weight \, there is an 
associated crystal graph C). We shall describe the structure of C) as a directed graph 
with colored edges. We mention that if g is a simple Lie algebra with root system 
®, and V) is the unique simple g module with highest weight A, then the quantized 
universal enveloping algebra Ug(g) has a corresponding module. A crystal base is 
a base for this module at q = 0. It carries a graph structure induced by its structure 
as a Ug(g) module. For further information, see [30]. Here we forgo exploring the 
connection with the quantum group. We instead explain the structure of a crystal 
as a colored directed graph and the parameterization of crystals by Berenstein— 
Zelevinsky—Littelmann paths and Littelmann patterns. 


2.3.1 The Crystal as a Colored Directed Graph 


We now describe Cy as a colored directed graph. Let B be a finite set, and we call 
elements of B elements (or vertices) of the crystal. (We shall abuse notation and write 
b €C) forab € B.) Forevery | <i <r, we have operators f; : BU {0} > BU {0} 
and e; : BU {0} + BL {0} acting on the vertices. We shall refer to these as root 
operators. They have the property that if b, b’ € B, then f;b = b’ and b = e;b’ are 
equivalent. This defines the structure of C, as a colored directed graph: if b, b’ € B 
and f;b = b’, then C) has a directed edge b > b’ “colored” by the index i. There 
is a weight function wt : B —> A such that wt(f;(b)) = wt(b) — a; and in fact the 
function wt is a bijection between B and a weight basis of the highest weight g-module 
V,. In particular, there is a unique “highest element” b) € C) with wt(b,) = A. This 
by is the unique element of B such that e;b, = 0 for every 1 <i <r. It follows that 

Bw aif fo fs | 1S ee lang, (10) 
We shall be interested in writing an element b € B as b = f7"' f/? --- f/"*by in par- 
ticular when the sequence of indices [i), i2..., iz] is a reduced word. 


2.3.2 Berenstein—Zelevinsky—Littelmann Paths 


Let w = [i1, i2..., ix] be areduced word in W, and letn = [11,n2..., ng] € (Z>0)* 
forb = fj" fi" --- fb). We call nan adaptive string of b [34] if forevery 1 < j <k 
we have 


l<j<k: ee fo fb, =0 (11) 


Uj+l Uk 
We can think of n as encoding a path from b to by along crystal edges (against the 
direction of the edges), using w as a road map. To get the path, starting at b we first 
take steps along edges colored i, as long as that is possible. After taking n; steps, we 
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arrive at a vertex bj = e'b such that e;,b; = 0. We then proceed with steps along 
edges colored i for as long as possible, etc. 

Taking an adaptive string above defines a map b +> Zs9'™ for any reduced word 
w. Let wy be a long word of the Weyl group. Write s SC Zso'™ for the set of 


adaptive strings that occur inC) and S,, © Zx9' 20 for the set of strings that occur for 
any strongly dominant \. Then it follows from work of Littelmann, Berenstein, and 
Zelevinsky [3, 33, 34] that Sy, is the set of integral points inside a convex cone, which 
we from now on refer to as the Littelmann cone C Wo . Furthermore, the set S%} A , 1s the 


set of integral points in a convex polytope C? in this cone (the Littelmann noliopel, 
The inequalities describing C,,, depend on the long word w,; the further inequalities 
describing C> , depend on A. as well. For particularly “nice” choices of wy [34], these 
inequalities take on a transparent form. We shall describe these choices for Cartan 
types A, B, C, and D as well as the Littelmann patterns they give rise to in Sect. 3. 
For w = Wg, we shall refer to the adaptive string n corresponding to a vertex b € C) 
(as well as the corresponding path in Cy) as the Berenstein—Zelevinsky—Littelmann 
path or BZL path of b and write BZL(b) = n. 


2.4 Multiple Dirichlet Series and Whittaker Functions 


We briefly introduce the objects from number theory that are produced by the con- 
structions in Sect. 4. Since we wish to focus on the combinatorics of the constructions, 
we keep the length of this section to a minimum. Our purpose here is merely to moti- 
vate the appearance of highest weight crystals as an apt combinatorial device in the 
study of these objects. 


2.4.1 Multiple Dirichlet Series 


Multiple Dirichlet series are series in several complex variables. They can be used 
to study automorphic L-functions, generalizations of the Riemann zeta function 
via the Langlands—Shahidi method. Of special interest to us here are Weyl group 
multiple Dirichlet series, whose functional equations are governed by a Weyl group 
associated to a (finite) Cartan type. The functional equations are of significance in 
proving meromorphic continuation and functional equations. We explain briefly how 
prime power coefficients of multiple Dirichlet series are related to sums over a highest 
weight crystal. We follow the notation of [17] with some simplifications, so as not 
to occlude the picture. 

Let F now be a global field. We wish to construct a series inr variables 51, ..., 5;: 


SB Cie Commi arom e(C | (12) 
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where the summation is over ideals C; of Os. Relating such a series to automorphic 
L-functions imposes certain restrictions on its construction. For example, though a 
series does not have an Euler product in the way the Riemann zeta function does: 


(oe) 


@=eo= 2 


1 = —s 
n=1 Pp prime P 


its coefficients satisfy a twisted multiplicativity and the series is hence determined 
by its p-parts 


CO 
YH Leper yap spre (13) 
kj=1 


where p is (a representative of) a prime ideal, and (/,, ..., /) correspond to a weight 
A= ba l; Wj. 

Constructing a Wey] group multiple Dirichlet series thus amounts to describing the 
coefficients H(p",..., p&; p"',..., p'") for any fixed weight \. Note that assigning 
a weight to every (kj, ...,k,) as above, we may interpret the p-part as a sum over 
the weight lattice A. Its support turns out to be finite, and in fact contained in the 
convex hull of the Weyl group orbit of 2. 

Recall that for a highest weight crystal C) (associated to a root system ®), the 
weight function wt : C, > A is a bijection between vertices of C) and a weight basis 
of a representation with highest weight 4. Hence the constructions of the p-part may 
be written as a sum over a highest weight crystal. 


2.4.2 Whittaker Functions 


Our aim here is to motivate why metaplectic analogues of the Casselman—Shalika 
formula lead to constructions involving highest weight crystals. 

Let G be a split reductive group defined over Z. (The reader may think of SL, or 
Sp>,-) Let F be a nonarchimedean local field (e.g., F = Q,, the p-adic numbers), 
and O c F the ring of integers in F (e.g.,O = Z,). LetG = G(F) and K = G(O) 
be a maximal compact in G. Let T C G be a maximal torus, and U C G be the 
unipotent radical of a Borel subgroup of G. (In the examples above, T is the group 
of diagonal matrices in G and U the group of upper triangular matrices with 1s on 
the diagonal.) Let G denote the Langlands dual of G (we have Shia = PGL, and 
SP>,. = SO,+1); let ® be the root system associated with G and A its weight lattice. 
(Here ® is of type A or type B for SL, or Sp,,, respectively.) To an element x € Ei: we 
may associate a Whittaker function W : G > C that satisfies W(ugk) = w(u)W(g) 
(foru € U,g € G,andk € K, where w is an unramified character of U). Let 7 € F 
be a uniformizer (e.g., p in F = Q,). By the Iwasawa decomposition, any element 
g € Gcan be written as g = unk, where u € U,k € K, and € A isacocharacter 
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of T. A Whittaker function W is then determined by its values W(x). In this 
classical (nonmetaplectic) setting, these values are determined by the Casselman— 
Shalika formula [19]: 


We, ) = [] d-9a7!x xa) (14) 


ae@et 


This expresses the values of a Whittaker function W(x, ) in terms of the character 
x(x) of a representation of G of highest weight A. (Here g = |O/7O| as before.) 
Now let n be a positive integer so that charF {n and |j12,| = 2n for the group 
Lon C F of nth roots of unity. Then an n-fold metaplectic cover G of G is a central 
extension 
11> hy) Gs oSi 


constructed from the Cartan datum of G and some arithmetic data on F [38]. By a 
metaplectic generalization of the Casselman—Shalika formula, we mean an analogue 
of (14) for Whittaker functions on G. As mentioned in Sect. 1.3, sucha generalization 
may produce such a formula as a sum over a highest weight crystal, or as a sum over 
a Weyl group. This is motivated by the shape of (14) and its similarity with the 
deformation of the Weyl character formula in (3). 


3 Littelmann Patterns 


We recall Littelmann patterns from [34] in each of the Cartan types A,, B,, C,, 
and D,. A pattern is an array of €(wo) nonnegative integers. Integral points of the 
Littelmann cone (see Sect.2.3) are in bijection with the set of patterns that satisfy 
a set of inequalities. Imposing a further set of inequalities gives a parametrization 
of integral points within the Littelmann polytope, i.e., a highest weight crystal for a 
fixed highest weight. The contribution of a single element to the sums in Sect. 4 will 
be phrased in terms of the corresponding Littelmann pattern. 

The branching properties of highest weight crystals and how it is reflected in 
the constructions will be made explicit in Sect.5. One may observe these branching 
properties in the extent to which the Littelmann patterns are consistent within an 
infinite family of Cartan types. Note also that the simple root a, that is “new” in rank 
r is associated only to entries in the top row of the pattern. 


3.1 The Choice of a Long Words 


Recall that a long word is a reduced decomposition of the long element of the Wey] 
group. The parametrization of crystal elements in terms of Littelmann patterns is 
dependent on the choice of a long word wy. The choice of particular “nice” long 
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words results in the Littelmann cone having a transparent description. We give the 
nice long words here for each infinite family of Cartan types. 

Notice that the choice is consistent within each family in the following sense. Let 
X stand for any of A, B, C, or D, and let we " be the choice of long word for type 


r 


X,, 1.e., rank r. Then the word wo " starts with the long word we "| from rank r — 1. 


The choices are as follows. 


i S10) C6. 2 aes OP Sd (15) 

i Sal SION) pe Trace es ee asen?)) (16) 
0 0 
D, 


w = [(), 2), G, 1,2,3),...,(,r—1,...,3,1,2,3,-...7] (17) 


3.2. The Shape of Patterns 


The choice of a long word wy establishes a bijection between elements of a crystal 
and £(w,)-tuples of nonnegative integers via BZL paths as in Sect. 2.3.2. We arrange 
these £(wo) integers as entries a;,; of a Littelmann pattern. The shape of the pattern 
reflects the choice of wy made. 

Each column of a pattern corresponds to a particular index | < j < r. Entries a;,; 
with the same column index j correspond to occurrences of same simple reflection 
in the word wo. A row of the pattern will correspond to a step in the rank within the 
infinite family of Cartan types. 

In the remainder of this chapter, we follow the convention that if a;,; is not an 
entry of a pattern, then a;,; = 0. (This is the case, for example, if i < 0 or j <i.) 


3.2.1 Type A; 


We have £(w") _ (we) =r forr > 2. A Littelmann pattern of type A, has r 
rows, with r — i + 1 elements in the ith row. We write £ = (a;,;)1<i<r and draw the 
i<j<r 


pattern aligned to the right: 


21 ,1]41,2)° °° |@1,r 


(18) 


42,2)° °° |@2,r 


ay,r 


348 A. Puskas 
3.2.2 Type B, and C, 


In this case, ¢ (wi") —¢ wy") = 2r — 1. These Littelmann patterns have r rows 


as well, but now the ith row has 2r — | entries, denoted a;,; fori < j < 2r —i. We 


write j = 2r — j and di,j = a;,;, and draw the patterns centered as follows: 


41,1]@1,2]* **\@,r]*  *\@1,2141,1 


(19) 


a2,2)- °° a2,r sae la2,2 


Gyr 


3.2.3. Type D, 

In this case, £ (wo’) ae (we) = 2r —2 forr > 3. The €(w?’) =r? —r inte- 
gers from a BZL path are now arranged into a Littelmann pattern with r — 1 rows. 
The ith row has 2r — 2i entries, qj,j fori < j < 2r — 1 —i. We use notation similar 
to type B and C and write j = 2r — 1 — j for a; = 4j,2,—-1_;. 


a1 ,1|@1,2}* * - 171i] °° 11,2141, 


(20) 


42,2) - + |d2,7\A2,r|+ + - |@2,2 


3.3 The Bijection with Crystal Elements 


We are ready to give the bijection between crystal elements and Littelmann patterns. 

Recall that the BZL path of a crystal element b consists of €(wy) segments. 
Taking the length of these segments produces a tuple BZL(b) = (1, ..., New,))- 
The entries of the Littelmann pattern £(b) corresponding to b are these integers n,, 
(1 <h < €(w)). The pattern £(b) is filled with elements of BZL(b) row by row 
proceeding from left to right and from bottom to top. For example, in type A, we 
have that £(b) = (aj, ;)1<i<r and: 

i<jsr 


Gyr = 11, Ap—-1,r-1 = 12, Ap—-1yrp = N3,---, Qi = NOC)+1 ee AP Nes) 


The shape of the Littelmann patterns above arranges entries in the same column if 
they correspond to the same edge label. We examine this property in more detail. 
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3.4 The Weight of a Pattern 


Let b be an element in a crystal element of highest weight A. Let BZL(b) = 
(1, ...,New,)) and £ = L(b) be the Littelmann pattern corresponding to b via the 
bijection above. Then the weight of b is easily recovered from entries of the pattern. 
Recall that the hth segment of the BZL path follows edges of the crystal labeled 
with index k = wy(h). These edges all correspond to a root operator for the simple 
root ax; i.e., they all have the same label k. It follows that 


w= ae: = Nh. (21) 
k=1 


Wo(h)=k 


The shape of the patterns has the following property. Entries in a single column 
of £(b) correspond to segments of the BZL path of b. These segments all run along 
edges of the crystal with the same color k (or ax) 1 < k < r. Figure | shows the index 
of the crystal edges corresponding to each column in the various types. Observe that 
reading off the index for elements in the top row gives the segment of w that is 
present in rank r but not in rank r — 1. 

We make this explicit for each of the infinite families. We define the weight 


s(£) = (s1,..., 5,) of a Littelmann pattern £ (5s, = s,(£)) so that: 
A —wt(b) = ) 7 se(£O)) + a (22) 
k=l 
3.4.1 TypeA 
In this case, £(b) = (;,;)1<i<j<-. A column consists of entries a),;,...,a;,;. If 
BZL(b) =(n,,..., Nere!y), then a;,; = Nes )4j-i and the corresponding segment of 


the BZL path of b lies along edges labeled r — j + 1. We define s,(£) forl <k <r 


Type A, Type B, or C, Type D, 


Fig. 1 Edge colors corresponding to columns of a pattern 
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(2) = 0 aire (23) 
i=l 


3.4.2 Type B or C 


Here we have £(b) = (aj, j)1<i<j<2r-i. For any 1 < k <r there are two columns 
corresponding to the edge index k, the one with j =r —k + 1 and the one with 
jJ=r—k-+1. We thus define: 


se(2) = YO Girtik + Gir4t-%) (24) 


i=1 


The middle column corresponds to the index | and so we define: 
n() => ay (25) 
i=l 


Note that |a| is different from |a2| = --- = |a,|. 


3.4.3 Type D 


This case is similar to the previous one. We have £(b) = (qj, ;)1<i<j<2r—1-i. For 
2 < k <r, the two columns corresponding to the edge index k are the jth where j = 
r —k + 1and jth, where recall that j = r — k. The two middle columns correspond 
to a; and a2, the roots on the “branched” end of the Dynkin diagram. Hence we 
define (cf. [22]): 


A (di, + dirs yif2<k<r 
s(£) = Dial = — (26) 
aa Gijr—2+k ifk = 1, 2 


3.5 Constraints on Littelmann Patterns 


The correspondence b +> £(b) described in Sect.3.3 above is a bijection between 
integral points of the Littelmann cone (see Sect.2.3.2) and the set of Littelmann 
patterns whose entries satisfy a certain set on inequalities, depending on the Cartan 
type of the underlying root system. To get a set of patterns in bijection with the 
integral points of a Littelmann polytope for \ (equivalently, a crystal of highest 
weight A), we may impose a further set of inequalities on the entries. This second set 
of inequalities shall depend on the highest weight .. In this section, we make these 
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constraints explicit for each of the infinite families of Cartan types. The constructions 
in Sect.4 phrase the contribution of a crystal element b in terms of whether these 
inequalities are satisfied by the entries of £(b) strictly or with an equality. 


3.5.1 Constraints for the Cone 


We give the inequalities describing Littelmann patterns corresponding to integral 
points of the Littelmann cone C. Let C*" denote the Littelmann cone in the Cartan 
type X,. Then we have the following. 


Theorem. [34] Let b correspond to £(b) under the bijection described in Sect. 3.3. 
Then b is an integral point of C*" if and only if the entries of £(b) are nonnegative 
and the following holds. 


X,=A,:  [34, Theorem 5.1] The rows are weakly decreasing: 
Git = in) Ze SG yr-1 = 4, foreveryl <isr (27) 


X,=8B,: [34, Theorem 6.1] For every row we have: 


24; = 24), i+1 ae ae 24} 7-1 247 = 24} r—| ee 2a; for every l<i<r 
(28) 
X,=C,: [34, Theorem 6.1] The rows are weakly decreasing: 
Gii = Qin De S Ay = iy = jy) = +++ = Gy foreveryl1 <i<r 
(29) 
X,=D,:  [34, Theorem 7.1] For every row we have: 
Gi = Git) Dee = Gir—2 = Gir-1, Gir = Gir-2 2° = Gi foreveryl<i<r—1 
(30) 


i.e., the rows are weakly decreasing with the exception of the central two elements. 
There is no restriction on the comparative size of these two elements. 


3.5.2 Constraints for a Polytope 


We introduce shorthand to refer to the sums of particular groups of elements of a 
Littelmann pattern. The notation s; ;(£), 5;,;(£), t,,-1(£), ti,,(£) used here differs 
slightly from that of [34] (s(aj,;), s(ai,;), etc.) to emphasize that s;,;(£) may be 
nonzero even if a;,; is not an element of the pattern £. When the pattern L is clear 
from context, we write s;,; for s;,;(£). 
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We define the following shorthand: 


Late if Sis type A 
Depa Ak, 5 + 7) if Lis type BorC,j <r—1 
51,j(Q) = d= ak, if Lis type Band j =r a1) 
a 2ak,r if Lis type C andj =r 
a (4e,j +%,;) if Lis type D, j <r —2 
a1 (@k,r—1 + x,y) if Lis type D, j =r- lorj =r 
Gi,j +5i-1,;(L) if Lis type Band j <r—-1 
or Lis typeC andj <r 
5, j(Q= or Lis type Dand j <r —2 (32) 
Gj,; + 2s;-1,;(£) if Lis type Band j =r 
5i,j)(&) if Lis type D, j =r—lorj=r 


Observe that 5; ; = 5; 7 when both are defined; when only one is, we use this to extend 
the definition. For patterns £ of type D, we shall also need: 


fir) = > aervand hy (S)= >) ae; (33) 


k=1 k=1 


We are ready to state the inequalities characterizing patterns that correspond to 
the integral points of a Littelmann polytope, or, equivalently, the points of a crystal of 
highest weight \. Let \ = )°;_, m; - w;. The integers m; appear in the inequalities. 

For type A,, the pattern £ corresponds to b € C) if the following inequalities are 
satisfied [34, Corollary 4]: 


i,j < M,-—j41 + 81,j-1(L) — 25j-1, (2) + si-1,j41(; forl<i<j<r (4 
For type B, and type C, the inequalities are as follows [34, Corollary 6.]: 


Gj, j < mp— joi +5), j7-1(L) — 25;-1, (+ 53-1, j41(0); forl <i < j <r —1(35) 
aij S Myp— ji +5), j7-1(©) — 25; (O +57, 74100); forl<i<j<r-1 (36) 
= my + d5;,-—1(£) — d5;_-1,-(£) (37) 


air 


where d = 2 in type B and d = 2 in type C. 
Finally for type D, the inequalities are as follows [34, Corollary 8.]: 


Gi, < My—j41 + Si, j-1(2) — 25;-1, (OD) + si-1,j41(Q); forl <i< j<r-—2 


(38) 
Gj < Myp—ji t+ $i, j41(D) — 25; (2) +5,,;-1(2); forl <i<j<r—2 (9) 
Gjp—1 SM + Sj,p-2(L) — 2t;-1,r-1(L) (40) 


Gir < my + Sj,~~-2(L) — 2t;-1,,(L) (41) 
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Let BZL(A) denote the set of Littelmann patterns £ that are in bijection with 
elements of the highest weight crystal C). 


4 The Constructions 


We are ready to give the constructions of p parts of WMDS and metaplectic Whittaker 
functions, i.e., the objects from Sect.2.4. To emphasize the combinatorial nature of 
the constructions in this section, we restrict our attention to constructing a polynomial 
P). The meaning of this polynomial in each of the types was given in Sects. 1.2 and 
2.4. 

We shall give a polynomial P for any Cartan type in the infinite families A,, B,, 
C,, and D,. 

The constructions are analogous in different types. Before giving the type-by-type 
constructions in Sect.4.3, we begin by summarizing the common elements. In this 
section and afterward, we shall identify a crystal element with the corresponding 
Littelmann pattern. 


4.1 The Contribution of a Pattern 


In all cases, P = P) is asum over a crystal of highest weight X. Using the bijection 
bt> L(b) above, we write P as a sum over BZ L(A). We shall have a sum: 


PS >) ox (42) 


LEBZL(A) 


where recall that C[A] was identified with a polynomial ring C[x]. Here wt(£) is 
essentially the weight of the pattern given in Sect. 3.4. 
The coefficient G(£) shall be given as a product: 


G2) =|] 9.5 (43) 
i,j 


This product is over elements of the Littelmann pattern, and the factor gj, ; (£2) depends 
only on the decoration of the element q;,; in £. For each of the infinite families, we 
decorate the elements of the pattern £ according to a circling and a boxing rule. 
An entry qj,; may be circled, boxed, neither, or both. Before giving the rules for 
decorating elements of £ in Sect.4.2, we preview how the decorations affect the 
contribution of L£. 
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In the case of the constructions in type A, B, and C, the factor gj, ;(£) depends 
only on the decoration of a;,; and the integer qj, ; itself.! In particular (in type A and 
cy: 

q“i ifa;,; is not boxed 


0 if a;,; is boxed (44) 


Ifa;,; is circled, then g;,;(£) := 


If a;,; is not circled, then the value of g;, ;(£) is a Gauss sum (see Sect. 2.2). 


4.2 Circling and Boxing Rules 


Recall that in Sect. 2.3.2, we identified elements of the highest weight crystal C, with 
integral points of the Littelmann polytope. In Sect.3, we gave a set of constraints 
that a pattern £(b) € BZL(A) satisfies if it corresponds to an element b € Cy. The 
constraints came in the form of inequalities (27)—(30) (these guarantee that b is in the 
Littelmann cone) and (34)—(38) (these are specific to the polytope and depend on 4). 
The decoration of an entry a;,; of £ depends on whether the inequalities involving 
qj,; are satisfied with an equality. 


4.2.1 Circling Rule 


The inequalities (27)—(30) involve a single row of the Littelmann pattern. An element 
dj,; appears in one of these, and that has a lower bound for a;,;. The lower bound is 
of the form: 


max (qj, j+1, i,;+2) if Lis of type D and j = r — 2 


Qj, j+2 if Lis of type Dand j =r —1 

Gj 2 5 i, j+1 if Lis of type Band j =r —1 (45) 
2+ di, j41 if Lis of type Band j =r 
i, j+1 otherwise 


Then aj, ; is circled if this lower bound (45) holds with an equality. 


'In type D, the picture is more complex. An analogous construction gives a result slightly different 
from the one expected from the p-part. The phenomenon of (symmetric) multiple leaners accounts 
for this discrepancy; see Sect. 4.3.4 for details. 

Note that in type B, we have the coefficient 1 instead of g“./. This discrepancy by a factor of q 
also present when comparing (47) with (46) or (48) can be eliminated with a change of variables 
in the polynomial P) (x). 
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4.2.2 Boxing Rule 


The inequalities (34)—-(38) have the property that every a;,; entry of a Littelmann 
pattern appears on the left-hand side of exactly one of them. Let a;,; be boxed if this 
inequality holds with an equality. 


4.2.3 Interpretation of Decorations 


Observe that an entry is circled or boxed when an inequality defining the Littelmann 
polytope is satisfied with an equality. This means that the element of the polytope 
corresponding to £ is on one of the hyperplanes defining the polytope. 


4.3, Constructions Type by Type 


We are ready to finish describing the constructions in type A,, B,, C,, and D,. 


4.3.1 TypeA 


We recall the definition of a p-part from [14]. First recall that the weight s(L) of 
a pattern was defined in (23) as the sum of entries in columns. In this case, we 
are interested in the p-part (13); we set P = P(x) equal to this p-part with \ = 
ye + 1)a; and we assume that A is a strongly dominant weight (m; = 1; > 1 
for every | <i <r). 

Then P is given by (42) where wt(£) = s(£) and G(X) is a product as in (43) 
where the factors of the coefficient are given as follows [14, Chapter 1]: 


0 if a;,; is circled and boxed 
sive qi ifa;,; is circled but not boxed 
aie g(ai,;) ifa;,; is boxed but not circled 


h(a;,;) ifa;,; is neither circled nor boxed 


(46) 


Here g = g; and h = h, are the Gauss sums from Sect.2.2 and q is the order of a 
residue field. 


Remark. This is the construction of the p part Hy from [14]. 
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4.3.2 TypeB 


Recall from Sects. 1.2 and 2.4.2 that [16] gives a conjectural formula for a Whittaker 
function in type B. In this case, we have that P, (x) is the value W(x, A) of a Whittaker 
function on a torus element. 

In this case, let wt(L) := a ae Sx — A where $(L) = (s1,..., 8,) as defined in 
(24) and (25). Then [16, Conjecture 2] states that P is given by (42) and (43) where 
the factors are given as follows: 


0 ifa;,; is circled and boxed 

1 ifa;,; is circled but not boxed 

q “4g: (aj,;) 1£a;,; is boxed but not circled 

q “ih,(qaj,;) 1£a;,; is neither circled nor boxed 


Kj§H = (47) 


where g, and h, are as in Sect. 2.2 and the subscript t is lif 7 = r andt = 2 otherwise. 


4.3.3. TypeC 


Once again we have P(x) be the p-part of a Multiple Dirichlet series from [2] or 
[26]. Let us once again write wt(£) = s(£). To specify P by (42) and (43), we must 
again specify the factors gj, ;(£) : 


qui ifa;,; is circled but not boxed 

9:(aj,;) if a;,; 1s boxed but not circled 

hy (a;,;) if a;,; is neither circled nor boxed and n|qj, ; 
0 otherwise 


Kj XH) = (48) 


where again t = 1 if j Ar andt = 2 is j =r. (We note that [2, (31)] contained a 
typo that was fixed by [26, (34)]: note that by (Sect. 2.2) if a;,; is neither circled nor 
boxed, then gj, ;(£) = 0 unless n|a;, ;.) 


4.3.4 TypeD 


Finally, we recall [22, Conjecture 1], a conjectural expression for the p-part P(x) 
of a Multiple Dirichlet series. In this case, the construction for P(= Py) is slightly 
different. Once again, it is a sum (42) over contributions from Littelmann patterns 
£ € BZL(A) and the weight of a pattern is wt(£) = s(L). The contribution G(L) 
of a pattern is again a product. However, in this case the factor g;,; is dependent on 
more than the decoration of q;,;. 

The definition of G(L) in [22] is written as a product over connected components 
of the decorated graph T'(£) of the pattern £. To give the conjectural construction 
of the p-part, we introduce some terminology. 
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The vertices of I (£) are the entries of £. Two entries belong to the same connected 
component in I'(£) if they are comparable in the inequalities (30) and they are equal. 
By the rightmost element of a component C, we mean the entry in C that is positioned 
rightmost in the Littelmann pattern. A connected component C is called a multiple 


leaner (m.1.) if it consists of entries aj,;, = dj, ;,41 = ++: = di,;, where j; <r — 2, 
r+1> jo and a; j,-1 > dij,j,, Gi,j, > 4, j+1- By the legs of C we mean the entries 
Gj, j, = +++ = dj,y—2 and aj,-41 = +++ = qj, ;,; the entry on the endpoint of the shorter 


leg of I’ is a;,;, or a;,;,. The component C is called a symmetric multiple leaner 
(s.m.].) if in addition j2 = j,; in this case, we define its length to be /(C) =r — ji, 
half the number of its vertices. 

We may write G(£) as in (43) but to define gj, ;(£), we write [22, 5.5] 


o(C)= [I gi,(2) (49) 


aj,jEC 


and give a(C) in terms of standard contributions of the entries it contains. Let [22, 
5.5] 
0 if the entry a is circled and boxed 
o(y) = 4 hi (a)-q~“ if the entry a is not boxed and not circled (50) 
gi(a)-q~“ if the entry a is boxed and not circled 


We then define a(C) to be as follows: 


e o(C) = Oif any a; ; € C is both circled and boxed. 

e o(C) = o(a) if C is nota m1. and a is its rightmost element, or C is a m.]. that is 
not symmetric and a is the endpoint of its shorter leg. 

e o(C) = a(a\ — gq") if C isas.ml., a ¥ 0 is its rightmost element, and a is 
unboxed. 

e o0(C) = 04, )o(Gi,j-1)4 
i,j # 0 is boxed. 

e o(C) = lif C isas.m.l. with zero entries. 


I if C is as.m.l., a;,; is its rightmost element, and 


Note that o(C) is a product over g; ;(£), but now g;,;(£) depends not only on the 
decoration of a;,; € £, but also on the position of a;,; within a connected component 
of (£2), and whether that component is a (symmetric) multiple leaner or not. 


5 Branching 


In the previous section, we described constructions of polynomials P(x) that are 
of interest from a number-theoretic perspective as explained in Sect.2.4. We also 
mentioned in Sect. 1.4 that elucidating the relationship of the constructions with 
the branching properties of highest weight crystals can be the key to understanding 
some of their properties. In this section, we take a closer look at how the branching 
properties of crystals manifests in these constructions. 
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In all of the examples above, the polynomial P was associated to a crystal Cy 
corresponding to a root system ® or rank r, with Cartan type in one of the infinite 
families A,, B,, C,, or D,. When the edges of C) labeled by a, are omitted, the 
remaining graph is a disjoint union of rank r — 1 crystals of the same Cartan type, 
but rank r — 1: 


Ga |NG (51) 
7 


We now wish explain how P) can be written in terms of the polynomials P,, associated 
to those crystals. 

The construction of P, is given (cf. (42)) as a sum over Littelmann patterns £ € 
BZL(A), each contributing a term G(£)x™'™). We examine how the sets BZ L(w) 
corresponding to y in (51) can be recovered from BZL(A) by giving a pattern 
L’ € BZL(u) for any pattern £ € BZL(A) in Sect.5.1. We indicate a method of 
computing the weights jy that appear in (51) in Sect.5.2. Then in Sects.5.2 and 5.3, 
we explain how for such pairs £ and £’ the contributions G(£) and wt(L) can be 
written in terms of G(L’), wt(L’) and yu. For the remainder of the discussion, let us 
fix a dominant weight A = )~);_, ma, and the corresponding crystal C). 


5.1 Patterns with Fixed Top Row 


Recall the bijection between elements of the crystal C, and Littelmann patterns given 
in Sect.3.3. For an element b € Cy, the entries of the pattern £(b) = £)(b) are the 
lengths of the segments in the BZL path of b. Here BZ L(b) corresponds to the choice 
of a particular long word wy. 

Let the element b € C) belong to C,, in the decomposition (51). We shall sometimes 
write b’ when we mean b as an element of the abstract crystal C,,; write £’ = L(b’) € 
BZL(w). 

As remarked in Sect. 3.1, the choices made in (15) all have the property that the 
long word we’ chosen in rank r starts with the long word wo "“! chosen in rank r — 1. 
Together with the shape of the patterns (see Sect.3.2), this means that £(b’) is the 
same as £(b) without its first row. This argument proves the following. 


Lemma. Let b, € C, C C) be the highest element within a crystal in the decomposi- 
tion (51). Let £,,(b) and £(b) denote the Littelmann patterns corresponding to any 
b €C, C Cy as an element of C,, and Cy, respectively. Then for any b € C,, C Cy the 
top row of &)(b) is the same as the top row of £)(b,,), and &,,(b) can be recovered 
from &)(b) by deleting the top row. 
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5.2 The Weights in the Decomposition 


In light of Lemma 5.1 and the inequalities on the top row of £ € BZL(A), we can 
describe the highest weights js appearing in the decomposition (51) by computing 
the weight of the highest element b,, € C,,. Recall that by (22) A — wt(b,) = A— 
can be expressed in terms of 5(£)(b,,)) : 


A= w= Yo s4(Lyy)) + % (52) 


k=1 


and furthermore that we have: 
$(£)(b)) = s(£)(b,)) + 5(£,.(b)) for any b € C, C Cy (53) 


Note that since the entries under the first row of £)(b,,) are all zero, the right-hand 
side of (52) can be written entirely in terms of the entries in the first row of £)(b,). 
The inequalities restricting the first row of a £ € BZL(A) involve no entries from 
any other row (cf. Sect. 3.5). It follows that given a highest weight \, we can recover 
the set of weights yz that appear in the decomposition (51). (This involves expressing 
the simple roots a, in terms of the fundamental weights, and carefully examining 
the restrictions on entries of the first row of a pattern £ € BZL(A).) We omit further 
discussion of this here and refer the reader to [14, (2.4)] for an example of a similar 
statement in Cartan type A. 


5.3. Branching and Contributions 


Let b be an element b € C,, C Cy. Let £ = £)(b) and L’ = £&,,(b). We wish to write 
P)(x) as asum 
Py(x) = DO p(w) « Pu) (54) 


Le 


where the weights yu are the ones of the decomposition (51) and p(j1) is a monomial. 

Before we explain why this decomposition is possible for the polynomials P)(x) 
we remark on the terminology of “branching.” Recall that the polynomial P\(x) can 
be thought of as a deformation of a highest weight character. Equation (54) has a clear 
analogue for highest weight characters. Let us write V, and V,, for the irreducible 
representations of highest weight \ and «u (inrank r andr — 1), respectively. If P, (x) 
were the character of V), then the coefficient in the monomial p(j:) would match the 
multiplicity of the V,, in the restriction of V) to a subalgebra of corank one determined 
by the first y — 1 simple roots. 
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The contribution of a pattern £ is of the form G(L)x"™ as in (42). The term 
x2) depends only on s(£). It follows from (53) that the first r — 1 component of 
s(£) is the tuple s(£’), while s,(£) = s,(£(b,,)) depends only on pu. 

We turn next to the coefficient G(£). Recall from Sect. 4.1 and in particular (43) 
that G({) is a product of factors gj, ;(£). The factor gj, ;(£) essentially depends on 
the decoration of an entry qj, ; in £, and the decorations in turn depend on whether the 
inequalities imposed on the entry a;,; by £ being an element of B ZL (A) are satisfied 
strictly or with an equality. It is immediate that the factors g,,;(£) corresponding to 
entries of the first row of £ depend only on yu. Closer examination of the inequalities 
imposed on the lower rows and the decorations show that in fact, gj, ;(£) can be 
written as a product of gj—1,;-1(£’) and a factor that depends only on jz, and not the 
element b € C,,. 

Thus we may conclude that one may in fact decompose P)(x) as in (54). For a 
precise statement of this flavor in Cartan type A, see [44, Proposition 16.]. 
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